Fostering Quality and Flow of Online Learning Conversations by Artifact-Centered Discourse Systems Abstract In this paper, we develop a theoretical model to argue that the quality and flow of online learning conversations vary greatly because students are continuously faced with the task of building and maintaining an adequate level of common ground in order to make sense of one another's epistemic activities. Given the costly high failure rate of computer-mediated communication system implementations (Arbaugh & Duray, 2002) , this is an important contribution because an asynchronous online discussion system that affords an insufficient degree of common ground incurs a high social interaction cost, which can reduce the level of epistemic activities in which students engage. We hope our theoretical contribution serves as a stepping-stone for investigating the underlying functional characteristics that may influence the quality and flow of asynchronous online discussions not only in higher education institutions, but also in corporate learning platforms.. In conclusion, this study is the beginning of an action design cycle that centers on how students use asynchronous online discussion systems, and consequently on how to design software systems that more effectively support collaboration and learning outcomes. Our stream of research is urgently needed with the growing number of graduate degree programs whose students are geographically dispersed. Toward this end, we are presently investigating the correlations between collaborative peer discourse and individual learning outcomes by using a fine-grained analysis method that further differentiates activities relating to building and maintaining common ground from epistemic activities. Moreover, we have re-designed the linked artifact-centered discourse system as an extension of this research. Our next steps will comprise experimental tests of visually directing students' annotations on more important parts of an academic text, such as central concepts, principles, and their interrelations. Reply from student F: Based on Clark's (1996) communication theory, this study examined the quality and flow of collaborative peer discourse facilitated by the functional characteristics of three asynchronous online discussion systems. In this section, we discuss why the functional differences stated at the outset led to the reported findings and then tie the findings to the proposed theoretical model and learning.. In response to the first research question, content analysis findings demonstrate that the presence of learning material fosters a positive shift in students' epistemic activities from experience-based messages to theory-based reflections. These findings confirm Van der Pol's (2009) result that an anchored discussion system facilitates more meaning-oriented collaborative peer discourse than a regular forum discussion. However, limited by the experimental design that compared the general effect of anchored discussion to regular forum discussion, Van der Pol (2009) could not attribute his reported result either to the presence of learning material or linking functionality. By isolating the presence of learning material from linking functionality in the parallel artifact-centered discourse system, our study attributes Van der Pol's (2009) reported result to the presence of the learning material. Moreover, the current findings go beyond Van der Pol's (2009) because they demonstrate that the mere presence of the learning material has no significant effect on how students construct arguments in order to make their points. This is important because the small number of grounded claims to justify perspectives and counterarguments to negotiate different perspectives can potentially undermine collaborative knowledge construction in both conventional and parallel artifact-centered discourse systems (see Ackerman & Halverson, 2004 , for a similar argument in the domain of knowledge management). A possible explanation for these findings is that argument quality encompasses factors such as clarity of phrasing and organization as pointed out by Wolfe (2008) , which are not related to the shared context provided by the online presence of the learning material.. Concerning the second research question, content analysis findings indicate that a bi-directional linking functionality increases the frequency of grounded claims to justify assertions and counterarguments to challenge learning partners' assertions. First, the reported increase in the frequency of grounded claims reinforces Pfister's (2005) argument that a bi-directional linking functionality that centers on shared annotations stimulates students to think about the relevance and merit of their thoughts before contributing them to collaborative peer discourse. Compared with other studies that evaluated students' online argumentation (e.g., Peters & Hewitt, 2010; Yeh & She, 2010) , our findings show a similar low occurrence of grounded claims. These findings are troubling because they indicate that doctoral-level students have difficulties in constructing warranted claims when discussing high-quality research papers online.. Second, the reported increase in the frequency of counterarguments suggests that a bi-directional linking functionality affords an adequate level of common ground, which, in turn, impacts the cognitive process of comparing one's own perspective with another's perspective. This process is in the foreground of a social constructivist perspective where the interaction of different perspectives is of primary concern. Importantly, this contribution means that a bi-directional linking functionality provides support in becoming aware of differences in individual perspectives. This kind of discourse has been shown to enhance individual learning outcomes because it helps students to look at the learning material from multiple perspectives (e.g., Eryilmaz, Van der Pol, Clark, Mary, & Ryan, 2010) .. With respect to the third research question, sequential analysis findings show that the presence of learning material keeps online discussion topic-focused because students are less likely to respond with personal experiences to messages that relate understanding of a theory to research questions.. Furthermore, it pushes students back toward understanding a research problem when they stray from the topic. A possible explanation for these findings is that placing learning material and its associated discussion in a single window increases the degree of common ground because students do not need to switch back and forth between two separate windows for a sustained on-topic discussion. While the presence of the learning material in the parallel artifact-centered discourse system suggests a stronger sense of common ground than the conventional discourse system, this effect may be bound to specific conditions, such as a student's comprehensive reference to a passage from the learning material and another student's attempt to access it (see Appendix C where two students attempt to discuss a proposition from a given research article).. Regarding the fourth research question, sequential analysis findings illustrate that a bi-directional linking functionality promotes students to re-think their initial understandings of a theory while examining both adequate and inadequate relationships between theory and research questions. This is an important finding because re-thinking helps students consolidate information and gain a deeper understanding of the learning material (Andriessen, Baker, and Suthers, 2003) . Moreover, a bidirectional linking functionality further re-focuses a discussion that has digressed into experience oriented messages. Taken together, these findings mean that a bi-directional linking functionality can extend the degree of common ground created without that process being bound to any specific condition (see Appendix D). This is an important contribution when conceptualizing success as the ability of a system to facilitate cognitive, on-topic, on-task, and sustained discussion because a bidirectional linking functionality can naturally create a strong context for topics that merit discussion.. Relating the obtained findings to individual learning outcomes, we can infer that learning occurs best when the degree of common ground affords articulation, reflection, and negotiation of different perspectives (as previously found by Kobbe et al., 2007) . This means that, when starting from an adequate level of common ground, students can build further new understanding by adding new relations and concepts to common ground via integration. In this way, students can use one another as a resource for learning (Dillenbourg, 1999) . However, in the absence of an adequate level of common ground, misunderstandings trigger disruptions in the flow epistemic activities. These misunderstandings, which Appendix C demonstrate, can make it difficult for students to benefit from different perspectives on a topic under discussion. This study's findings indicate that, under such conditions, students state their own perspectives, but do not further develop them in light of other students' contributions (see Eryilmaz, Van der Pol, Clark, Mary, & Ryan, 2010 , for a similar finding).

O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Chronic

disease

modeling

and

simulation

software

Computers

allow

describing

the

progress

of

a

disease

using

computerized

models.

These

models

allow

aggregating

expert

and

clinical

information

to

allow

researchers

and

decision

makers

to

forecast

disease

progression.

To

make

this

forecast

reliable,

good

models

and

therefore

good

modeling

tools

are

required.

This

paper

will

describe

a

new

computer

tool

designed

for

chronic

disease

modeling.

The

modeling

capabilities

of

this

tool

were

used

to

model

the

Michigan

model

for

diabetes.

The

modeling

approach

and

its

advantages

such

as

simplicity,

availability,

and

transparency

are

discussed.

Recent

interest

in

disease

forecast

has

resulted

in

a

multitude

of

disease

models

with

competing

strengths

and

weaknesses.

For

example,

most

disease

models

are

dedicated

software

tools

in

a

variety

of

languages

that

are

developed

for

the

sole

purpose

of

simulating

a

specific

disease

model.

The

UKPDS

outcomes

model

is

embodied

within

an

XL

workbook,

the

global

diabetes

model

is

implemented

in

Visual

Basic,

and

the

eagle

diabetes

model

is

coded

in

C++..

These

models

use

Monte-Carlo

simulation,

which

is

the

simulation

engine

that

is

generalized

and

implemented

by

our

software.

The

CDC/RTI

model

was

designed

using

a

Markov

model,

which

our

tool

also

allows.

All

of

these

tools

are

limited

to

diabetes,

this

is

reasonable

considering

the

huge

cost

involved

in

developing

the

model..

In

contrast,

the

Archimedes

model

is

different

from

the

above

models

in

that

it

uses

an

object

oriented

approach

to

model

progression

of

continuous

variables.

Archimedes

has

been

published

extensively,

yet

being

proprietary

and

of

a

different

mathematical

framework,

it

is

conceptually

different

than

our

open

framework..

Similar

to

our

framework,

some

diabetes

models

have

an

associated

GUI.

The

Global

diabetes

model

has

an

interface

that

allows

defining

population

parameters

before

simulation.

Another

notable

user

interface

is

the

Diabetes

Ph.D.

web

site

that

uses

the

Archimedes

simulation

engine.

However,

these

interfaces

are

limited

to

managing

values

of

model

specific

parameters

and

model

structure

can

not

be

changed.

It

is

also

not

possible

to

define

a

new

model

using

these

user

interfaces..

In

contrast,

our

framework

can

accommodate

and

manipulate

multiple

distinct

models

and

it

is

not

dedicated

to

a

single

model.

This

is

a

major

difference

between

our

software

framework

and

other

modeling

tools.

For

example,

our

software

can

be

used

to

develop

disease

models

other

than

diabetes.

Furthermore,

by

modifying

our

model

structure,

diabetes

researchers

can

use

our

existing

diabetes

model

to

project

outcomes

based

on

the

results

of

their

own

innovative

research.

We

consider

this

to

be

an

important

benefit

of

our

general

simulation

engine

and

GUI..

However,

disease

models

are

not

limited

to

diabetes.

During

the

work

on

our

project

we

encountered

disease

models

and

related

tools

for

various

other

diseases,

including

Influenza

Coronary

Heart

Disease

and

Cancer

This

is

just

a

partial

list

that

demonstrates

the

variety

of

dedicated

tools

available.

To

our

knowledge

to

date

there

is

no

other

software

framework

that

is

flexible

enough

to

accommodate

multiple

chronic

disease

models

that

has

the

following

capabilities:

The

ability

to

define

the

model

structure,

the

ability

to

define

parameters

and

formulas,

the

ability

to

set

simulation

rules

and

perform

simulation,

the

ability

to

analyze

the

output

data..

These

simulation

capabilities,

offered

by

our

framework

are

general

enough

to

model

chronic

diseases.

Yet

our

framework

is

not

suitable

for

modeling

infectious

diseases

since

these

require

interaction

between

individuals,

which

our

framework

does

not

currently

support.

Also,

our

support

does

not

extend

to

general

continuous

time

modeling

as

we

are

confined

by

a

constant

length

simulation

step.

Since

our

simulation

engine

includes

event

states,

it

already

handles

event

queues,

which

can

be

extended

further

in

future

research.

Another

gap

in

our

framework

is

that

our

simulation

capabilities

are

more

advanced

than

the

estimation

capabilities.

Estimation

of

model

parameters

is

currently

confined

to

a

Markov

model

in

a

single

process.

Expanding

estimation

capabilities

is

currently

a

research

topic

of

much

interest.

Possible

research

directions

include

multiple

subprocesses,

and

model

expression

support.

To

allow

others

to

participate

in

this

research

effort,

we

are

publishing

our

methodology

and

we

have

published

our

estimation

source

code

both

in

Matlab

and

Python

environments..

Publishing

the

disease

modeling

software

for

free

along

with

its

open

source

is

compatible

with

the

design

of

the

modeling

approach

that

uses

simpler

more

intuitive

techniques

to

model

diseases.

These

intuitive

approaches

reduce

the

learning

curve

for

researchers

who

are

entering

the

field.

It

also

enables

collaboration

between

researchers

specializing

in

different

facets

of

a

single

disease.

Finally,

it

enables

researchers

to

develop

new

disease

models

without

recreating

a

GUI

or

a

computation

engine.

The

tool

exists,

and

clinical

collaborators

can

focus

their

efforts

on

clinical

matters

rather

than

technical

matters..

Another

advantage

of

our

software

framework

is

that

with

our

relatively

simple

modeling

techniques,

we

support

a

wider

human

comprehension

of

disease

processes.

Using

standardized

"base"

models

of

the

natural

history

of

disease,

researchers

can

incorporate

the

effect

of

proposed

interventions

into

the

base

model

and

simulate

expected

long-term

outcomes

of

early

intervention.

This

not

only

gives

feedback

to

researchers

about

the

utility

of

their

preventive

actions;

but

it

provides

a

motivation

to

target

transitions

in

a

model

where

an

intervention

will

have

the

most

significant

effect.

Then,

as

studies

are

conducted,

that

data

may

be

used

to

calibrate

and

correct

the

base

model

as

the

state

of

the

science

advances.

Together,

these

and

other

practical

uses

of

the

software

will

help

researchers

comprehend

complex

disease

processes.

Making

this

tool

available

freely

and

openly,

facilitates

disease

modeling

activities

and

therefore

contributes

to

better

understanding

of

disease

progression..

Not

only

can

researchers

benefit

from

a

disease

modeling

software,

we

hope

that

in

the

future

physicians

can

eventually

use

the

software

as

an

educational

tool.

Working

with

a

patient,

a

physician

can

simulate

patient

outcomes

based

on

the

demographic

characteristics

of

a

specific

patient,

for

a

variety

of

proposed

treatment

plans.

In

this

manner,

a

patient

can

become

a

more

active

participant

in

his

or

her

own

health

care

decisions..

Our

web

site

also

contains

additional

information

on

the

project,

beyond

the

published

software

and

diabetes

model.

The

web

site

can

be

accessed

online

at:

FIGURE

The

Michigan

model

for

diabetes.

IGT

means

Impaired

Glucose

Tolerance,

ESRD

means

End

Stage

Renal

Disease,

and

MI

means

Myocardial

Infarction.

Flow

diagram

of

the

Monte-Carlo

simulation

An

example

of

the

project

form.

This

form

allows

defining

the

model

and

population

set

to

be

used

as

well

as

simulation

parameters

such

as

the

number

of

simulation

steps

and

the

number

of

repetitions.

The

form

allows

manipulating

rules

according

to

their

phase

in

the

simulation.

Each

phase

is

displayed

in

a

different

tab

for

the

convenience

of

the

user.

Table

Sample

output

from

simulation

results.

This

table

shows

selected

result

columns

(out

of

for

a

few

individuals.

Time

means

the

initial

baseline

population

as

generated

by

the

system

before

simulation..

State

indicator

values

are

set

to

if

the

individual

is

in

that

state

during

a

certain

time

step.

Note

that

entering

a

state

raises

the

state

indicator.

The

utility

of

a

system

such

as

described

in

this

paper

depends

heavily

on

the

feasibility

and

validity

of

the

models

used.

To

demonstrate

the

system

feasibility,

we

have

created

many

simulation

tests

that

are

published

with

the

software.

This

first

level

of

validation

demonstrates

correctness

and

serves

for

quality

assurance..

To

demonstrate

applicability

to

disease

modeling,

we

have

developed

the

Michigan

model

for

diabetes

using

our

software

framework.

This

model

includes

disease

subprocesses

with

about

states

and

about

transitions

as

seen

in

Figure

More

than

variables

and

more

than

simulation

rules

are

used

by

this

model.

The

model

went

through

more

than

versions

during

its

development

process.

The

development

process

included

the

following

stages:

literature

review;

model

formulation;

validation

of

our

model

against

clinical

study

results.

These

development

stages

were

repeated

until

convergence

to

the

published

model

version

was

achieved.

The

model

file

titled

"Michigan

model

for

diabetes

can

be

downloaded

from

the

model

section

of

web

site

using

the

following

link:

This

model

file

is

a

compressed

archive

that

contains

a

documentation

file

that

details

the

above

development

stages:

The

literature

used

to

create

the

model

is

provided

in

the

bibliography;.

The

manual

calculations

used

to

extract

the

transition

probabilities

from

the

literature

are

described

in

great

detail

so

these

can

be

replicated

by

others;

and

The

model

validation

against

the

UKPDS

results

are

summarized

in

more

detail

than

will

be

presented

below..

To

allow

model

validation,

a

population

of

individuals

was

artificially

generated

from

distributions

provided

by

the

population

as

described

in

section

This

population

was

used

as

a

baseline

population

for

the

model

simulation

the

results

of

which

were

validated..

Model

validation

was

conducted

by

repeating

the

model

simulation

for

times.

The

simulation

process

took

about

hours

on

processor

cores

(about

days

on

a

single

processor

core).

At

the

end

of

the

simulation

process,

a

large

array

of

records

was

generated.

Each

record

contained

the

values

of

all

model

parameters

for

each

person

in

each

living

year

for

each

repetition.

An

example

of

such

output

is

presented

in

the

appendix.

These

simulation

results

were

analyzed

and

compared

against

UKPDS

results

appearing

in

Table

shows

selected

simulation

results

compared

to

UKPDS

aggregate

results.

This

validation

shows

a

good

fit

in

most

categories.

The

micro-vascular

category,

as

defined

by

UKPDS

includes

elements

from

the

retinopathy

process,

which

we

recognize

needs

improved

modeling.

This

is

an

area

of

active

research..

Our

diabetes

model

used

parameters

based

on

the

UKPDS.

For

example

we

calculated

death

rates

from

MI

based

on

published

rates

in

the

UKPDS

Simulation

results

were

tabulated

to

comparable

summary

measures

as

reported

in

figure

in

Our

report

engine

allowed

proper

counting

of

incidences

and

of

persons

affected

by

incidences

for

each

category

for

specified

periods.

More

specifically,

using

our

software,

it

is

possible

to

count

the

number

of

persons

who

had

a

stroke

in

the

first

years,

or

the

total

number

of

strokes

in

the

population

in

the

same

time

period.

Our

framework

also

calculates

statistics

of

repetitions.

The

results

in

table

were

calculated

using

these

mechanisms..

To

allow

further

and

more

detailed

scrutiny

of

our

work,

these

validation

results

along

with

a

fully

documented

diabetes

model

are

available

online

at

our

web

site.

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O


Random-order bin packing The average case analysis of algorithms usually assumes independent, identical distributions for the inputs. In [?], Kenyon introduced the random-order ratio, a new average case performance metric for bin packing heuristics, and gave upper and lower bounds for it for he Best Fit heuristics. We introduce an alternative definition of the random-order ratio and show that the two definitions give the same result for Next Fit. We also show that the random-order ratio of Next Fit equals to its asymptotic worst case, i.e., it is 2.

O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Three efficient numerical models to analyse the step problem in shallow water In this paper, the problem of acoustic wave propagation in a waveguide of infinite extent is modelled, taking into account constant depth in each section of the sea. Efficient In the numerical models only the inclined or vertical interface between the sub-regions of different depth are discretized, as Green's functions that take into account the presence of free and rigid surfaces are used. These Green's functions are obtained either by eigenfunction expansion or by Ewald's method. A detailed discussion on the performance of these formulations is carried out, with the aim of finding an efficient numerical formulation to solve the step problem in shallow water. Many analytical and numerical methods have been developed to simulate and analyse underwater acoustic wave propagation. The book by Jensen et al. [1] discusses in detail the different methodologies applied to solve the problem of wave propagation in acoustic environments that have interested many researchers over past decades. Some of the wellknown methods are based on the acoustic ray theory [2] , the normal modes method [3] and the parabolic equation [4] .. A variety of numerical models has also been developed based on well-established approaches such as the finite difference, finite element and boundary element methods. Of these, the Boundary Element Method (BEM) permits an efficient analysis of underwater acoustic problems with complex shapes and complicated boundary conditions. The BEM has a number of advantages over other numerical methods [5] , such as: it is very well suited for modeling homogeneous unbounded domains since it automatically satisfies the Sommerfeld radiation condition and thus involves a more compact description of the acoustic medium, requiring only the discretization of the problem boundaries, which considerably reduces the size of the final linear system of equations. However, the application of the boundary integral equation is often limited by the requirement of prior knowledge of the fundamental solutions and the appearance of singular or hyper-singular integrals in its formulation.. Another difficulty of the BEM in the analysis of acoustic wave propagation in shallow water occurs when more complex geometries are considered, requiring large discretization schemes. One way of avoiding this large discretization is by using Green's functions which directly satisfy the boundary conditions on the flat free surface and the rigid bottom of the ocean. Such Green's functions can be constructed using the image-source technique, but this leads to very slowly convergent series [6, 7] . An alternative to improve the convergence of the series is to build a Green's function in the form of eigenfunction expansions, the so-called normal mode solution. This function is also an infinite series but if only the evanescent modes are considered and there are no propagating modes, the series becomes rapidly convergent owing to the exponentially decaying terms for the evanescent modes. In spite of that, the convergence problems of this series still remain when the source and the field points are positioned along the same vertical alignment [8, 9] .. Linton [10, 11] and Papanicolaou [12] discuss mathematical techniques for accelerating slowly convergent series and show that the Ewald's method is able to provide dramatic improvements in the speed of convergence, particularly when the source and field points are located along the same vertical line. This method has been successfully implemented in the boundary integral equation formulation by Venakides et al. [13] , for the calculation of electromagnetic scattering of photonic crystals.. Santiago and Wrobel [8, 9] discussed the implementation of Ewald's method in a BEM model for underwater acoustics. They compared the convergence of Ewald's method with that of eigenfunction expansions, showing a substantial reduction in the number of terms necessary for convergence of the series, particularly when the source and field points are positioned along the same vertical line. In the present paper, for the first time, Ewald's method is fully implemented in a BEM underwater acoustics model in which a vertical interface is discretized, significantly improving the performance of the method. The integration of the resulting singular integrals is also discussed in the paper.. In recent years, meshless methods have attracted great interest of scientists and researchers. The Method of Fundamental Solution (MFS) is one of these methods and it has been applied with success for scattering or radiation problems. Mathematically, the MFS is a very simple technique and it is also based on the prior knowledge of fundamental solutions, but not requiring the numerical and analytical integrations that need to be performed in the BEM. One disadvantage of the MFS is the determination of the position of the pseudoboundary on which the singularities are placed. Karageorghis [14] has proposed a simple algorithm for estimating an optimal pseudo-boundary for certain boundary value problems.. Costa et al. [15, 16] have shown that, despite its simplicity, the MFS is a very interesting tool to efficiently predict wave acoustic propagation in shallow water.. In this paper, the Boundary Element Method and the Method of Fundamental Solutions are used to analyse, in the frequency domain, the two-dimensional acoustic wave propagation in a shallow water configuration, considering a step up on the bottom of the sea. Time domain signals are computed by means of an inverse fast Fourier transform of the numerical results in the frequency domain. Appropriate Green's functions are used limiting the number of discretized surfaces and consequently reducing the computational cost of the proposed models. These models are developed by using a sub-region technique, where only the inclined or vertical interface between the sub-regions of different depth has to be discretized. These Green's functions are obtained either by eigenfunction expansion or by Ewald's method. A set of numerical examples is performed in order to demonstrate the efficiency of the proposed models in the analysis of acoustic wave propagation in shallow water problems containing a step on the seabed. In addition, a detailed discussion on the performance of these formulations is carried out, with the aim of finding an efficient formulation to solve the acoustic step problem in shallow water in the frequency domain.

O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I I O O B I I I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O


Topological characterization of neuronal arbor morphology via sequence representation: I - motif analysis


Rendering traditional mosaics This paper discusses the principles of traditional mosaics and describes a technique for implementing a digital mosaicing system. The goal of this work is to transform digital images into traditional mosaic-like renderings. We achieve this effect by recovering free-form feature curves from the image and laying rows of tiles along these curves. Composition rules are applied to merge these tiles into an intricate jigsaw that conforms to classical mosaic styles. Mosaic rendering offers the user flexibility over every aspect of this craft, including tile arrangement, shapes, and colors. The result is a system that makes this wonderful craft more flexible and widely accessible than previously possible. We have presented a technique for rendering traditional mosaics. The proposed system exploits extracted and drawn feature curves to generate a tessellation in which to lay down tiles. The colors of the tiles are sampled from the underlying image. We reviewed the use of Voronoi diagrams to help trim the offset curves. A fast algorithm based on a hardware Z-buffer was used for this purpose [13] . Although the use of Voronoi diagrams has been suggested in the literature to produce mosaic images, it is important to note that we use Voronoi diagrams only as a means to trim offset curves. The straightforward application of Voronoi diagrams for tessellation, as is used in some commercial software, does not produce tile placement consistent with traditional mosaics. While this paper lays out the basic approach to perform traditional mosaicing, a number of enhancements are possible. Future work remains in implementing the ideas described below. The presented approach leaves visual artifacts in the forms of virtual lines along the skeleton of the feature curves, or the locus of singular points of the offset curves. Further, due to the precise nature of the proposed approach, the placement of the tiles seems too precise at times. One can expect that by employing a gradient field, proposed by [12] , this synthetic artifact can be mitigated. Here, tiles in the neighborhood of the skeleton will be allowed to move under the influence of the gradient field an amount that is a function of the distance to the skeleton, thereby preserving the overall precise placement of tiles. Visually interesting effects are possible if we use tiles textured with another image. This idea is actually a generalization of photomosaics [25] . In photomosaics, an abstract version of an image is generated by packing a dense set of smaller images into a nonuniform upright grid. In mosaics, we may abandon the rectilinear grid with intricately flowing rows of tiles, each consisting of a small image. It is also possible to scan a set of mosaic tiles to create a library of interesting tile textures. Then, the underlying color of the digital image in the tile neighborhood can be applied to tile textures randomly selected from the library to produce a visually interesting effect.. Although traditional mosaics place tiles along rows that grow from designated feature curves, it seems reasonable to consider other tile placement strategies. Indeed, the digital engraving system described in [22] shares similar goals for orienting lines in response to image features. This suggests the use of engraving lines, as produced by that system, for tile placement. The mosaic image need not be constructed from flat 2-D tiles. Instead, we can render the image in 3-D with any desirable reflection model. Interesting effects are possible if we integrate the stone-generation algorithm of [20] to render the mosaic tiles. It is interesting to note that during the Byzantine era, mosaic tiles were often set at oblique angles to achieve a shimmering effect. This effect can be simulated easily once we impose specular reflection on the 3-D tiles.. The proposed system deals primarily with rectangular and triangular tiles. A future enhancement will permit the user to introduce other tile shapes, including hexagons, diamonds, and irregular shapes. There are several benefits to a digital mosaic rendering system. Every aspect of the creative process can now be edited and rendered at near real-time rates. The user may readily alter the set of feature curves, edit the offset curves, designate the composition rules, and control the tile shapes and colors. The result is a system that makes this wonderful craft more flexible and widely accessible than previously possible. A key observation derived from traditional mosaics is the fact that they emphasize feature curves of importance in the picture. Rows of tiles are arranged along these feature curves. Here, two typical arrangements can be found: the rows along feature curves continue throughout the picture, or alternatively a small number of rows follows the feature curves whereas a background tiling covers the rest of the picture. In order to emulate the placement of mosaic tiles along feature curves, we seek a model that is able to compute arrangements of tiles along freeform curves. An algorithm that is capable of such a requirement must obey the following steps:. • Detecting and extracting feature curves from the picture. This stage is described in Sect. 2.1.. • Computing the offsets of these feature curves for all the rows of tiles that are expected to follow this feature curve. The offset computation procedure is presented in Sect. 2.2.. • Trimming the offset curves in self-intersecting locations so the tiling is indeed proper. The problem, as well as the computation of properly trimmed offsets, is described in Sect. 2.3.. • The placement of the tiles of the mosaics. This process is presented in Sect. 2.4.

O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O B I O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O


Some applications of lattice based root finding techniques Abstract. In this paper we present some problems and their solutions exploiting lattice based root finding techniques. In CaLC 2001, Howgrave-Graham proposed a method to find the Greatest Common Divisor (GCD) of two large integers when one of the integers is exactly known and the other one is known approximately. In this paper, we present three applications of the technique. The first one is to show deterministic polynomial time equivalence between factoring N (N = pq, where p > q or p, q are of same bit size) and knowledge of q −1 mod p. Next, we consider the problem of finding smooth integers in a short interval. The third one is to factorize N given a multiple of the decryption exponent in RSA. In Asiacrypt 2006, Jochemsz and May presented a general strategy for finding roots of a polynomial. We apply that technique for solving the following two problems. The first one is to factorize N given an approximation of a multiple of the decryption exponent in RSA. The second one is to solve the implicit factorization problem given three RSA moduli considering certain portions of LSBs as well as MSBs of one set of three secret primes are same. It is well known that given two large integers a, b (a > b), one can calculate the GCD efficiently in O(log 2 a) time. In [HOW01] , Howgrave-Graham has shown that it is possible to calculate the GCD efficiently when some approximations of a, b are available. This problem was referred to as Approximate Common Divisors problem. Coron and May [COR07] used the strategy of [HOW01] to proved the deterministic polynomial time equivalence of computing the RSA secret key and factoring. In this paper we present three other interesting applications of the technique presented in [HOW01] .. First, we use the idea of [HOW01] to prove that factoring N is deterministic polynomial time equivalent to finding q −1 mod p when p > q or p, q are of same bit size. In the presentation of a recent paper [HEN09] at Crypto 2009, it has been asked how one can use q −1 mod p towards factorization of N as q −1 mod p is stored as a part of the secret key in PKCS #1 [PKCS] .. In this direction, let us briefly explain RSA [RSA78] first. One needs to generate two large primes p, q, with (in general) q < p < 2q. Then we have N = pq and φ(N ) = (p − 1)(q − 1). Further, e, d are identified such that ed = 1 + kφ(N ), k ≥ 1. N, e are publicly available and the plaintext M ∈ Z N is encrypted as C ≡ M e mod N . The secret key d is required to decrypt the ciphertext as M ≡ C d mod N .. To make the decryption process faster, the Chinese Remainder Theorem has been exploited and the model is well known as CRT-RSA [QUI82, WIE90] . The encryption technique is same as RSA, but the decryption process is little different. Instead of one decryption exponent as in standard RSA, two decryption exponents (d p , d q ) are required in this case, where d p ≡ d mod (p − 1) and d q ≡ d mod (q − 1). To decrypt the ciphertext C, one needs to calculate both C p ≡ C dp mod p and C q ≡ C dq mod q. From C p , C q one can get the plaintext M by the application of CRT using q −1 mod p. This is the reason, q −1 mod p is stored in the secret key part of PKCS #1 [PKCS] .. One may be tempted to consider the following method to factorize N from the knowledge of q −1 mod p, which does not actually work. Consider q 1 = q −1 mod p. Given N , one can easily calculate q 2 = q −1 1 mod N using Extended Euclidean Algorithm. Now q 2 q 1 − 1 is divisible by N and hence q 2 q 1 − 1 is divisible by p. Thus, q 2 ≡ q −1 1 mod p ≡ q mod p. If q 2 would have been less than p, then q 2 = q and the factorization will be immediate. However, in general, q 2 is of O(N ) and not less than p. Thus this method does not work and we need to look for a lattice based strategy which we explain in Section 2.1.. Next we consider the problem of finding smooth integers in a small interval [BON00] . Finding smooth numbers is important for application in the well known factorization algorithms such as quadratic sieve [POM84] and number field sieve [LEN93] . We study the results of [BON00] and show that slightly improved outcome could be achieved using a different strategy following the idea of [HOW01] . This is presented in Section 2.2.. The paper [RSA78] itself presents a probabilistic polynomial time algorithm that, on input N, e, d, provides the factorization of N . It has been proved that [MAY04, COR07] given N, e, d, one can factor N in deterministic poly(log N ) time provided ed ≤ N 2 . In Section 2.3, we consider a slightly different scenario when a multiple of d say µd is known, but d is not known. Given that µ is very large, it is not possible to factorize µd easily. Thus to factorize N in such a scenario, we need to consider different approach. We exploit the idea of [HOW01] again and prove that given µd, e, N , one can factor N provided µ < N . The time complexity of our deterministic algorithm is e · O(log N ). Indeed, the algorithm is not feasible for large e. However, the most popular mode of RSA considers small e, say of the order of 2 16 + 1. In such a scenario, the algorithm works in poly(log N ) time.. To extend the problem little further, in Section 3.1, we assume that instead of µd, some approximation of it is available. We note that the idea of finding roots of a polynomial as described in [ELL06] can be suitably exploited here.. Finally, in Section 3.2, we study the implicit factorization problem introduced in [MAY09] . For our purpose we need the following two results. We first state the following one due to Howgrave-Graham [HOW97] .. be the sum of at most ω monomials. Suppose that h(. We also note that the basis vectors of an LLL-reduced basis fulfill the following property [LLL82] .. Lemma 2. Let L be an integer lattice of dimension ω. The LLL algorithm applied on L outputs a reduced basis of L spanned by {v 1 , . . . , v ω } with. in polynomial time of dimension ω and the bit size of the entries of L.

O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B I O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O B I O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Between entropy and subentropy The von Neumann entropy and the subentropy of a mixed quantum state are upper and lower bounds, respectively, on the accessible information of any ensemble consistent with the given mixed state. Here we define and investigate a set of quantities intermediate between entropy and subentropy. We have identified and studied various functions that lie between the entropy S and the subentropy Q. Our first set of such functions R (n) r emerged as a natural mathematical generalization of Eqs. (4) and (5), and also turned out to be generalizations of the alternative expression (13) for Q as an integral over the probability simplex. These functions share certain properties with entropy-they are concave, they take the value zero when all but one of the eigenvalues of ρ are zero, and they take their maximum value when all the eigenvalues are equal-but unlike entropy they do not remain unchanged when one includes additional dimensions corresponding to zero eigenvalues of ρ.. The related functions R α are weighted averages of the R (n) r 's and therefore share the properties just listed, but in addition they are invariant under the inclusion of null dimensions. Moreover they are the most basic functions having this property: other augmentation-invariant functions can be obtained as convex combinations of the R α 's.. One consequence of this invariance is a very modest kind of additivity. Let ρ 1 be an arbitrary density matrix of some quantum system and let ρ 2 be the density matrix of a pure state of another system. Then for any α in the range 0 ≤ α ≤ 1, we can say. This statement follows from the augmentation-invariance of R α along with two simple facts: (i) ρ 1 and ρ 1 ⊗ ρ 2 have the same nonzero eigenvalues, and. (ii) R α (ρ 2 ) = 0. On the other hand, for arbitrary ρ 1 and ρ 2 , R α is not additive except when α = 0, in which case R α is the entropy itself. Does either R (n) r or R α have a physical meaning? At this point we have no definite interpretation of either of these quantities, though because of its nice mathematical properties we have more hope for R α . Here we suggest one way in which this quantity might play a role in quantum information theory.. Consider once again an ensemble E = {(|ψ i , p i )} of pure states of a quantum particle, and suppose that one is trying to convey classical information by sending a sequence of states chosen from this ensemble, with frequencies of occurrence asymptotically equal to the given probabilities p i . If the receiver (Bob) is required to measure each particle individually, then the maximum amount of information that the sender (Alice) can convey per particle is the accessible information of the ensemble E. Suppose, though, that Bob is able to measure pairs of particles jointly. Then Alice can hope to convey more information per particle by encoding her message in codewords consisting of pairs of the original states; that is, each codeword is of the form |ψ i 1 ⊗ |ψ i 2 with |ψ i 1 and |ψ i 2 chosen from E. We insist that Alice respect the original probabilities of E in the sense that in a long message, each state |ψ i is used with a frequency approximating p i . One finds that Alice often can increase the information conveyed per particle by using this strategy [10, 11, 12, 13] . Moreover, by continuing to increase the length of the codewords, assuming that Bob can make arbitrary joint measurements on a whole codeword, Alice can convey even more information. Let I m be the amount of information one can convey per particle when the codeword length is m. The limiting value of I m for arbitrarily long codewords is simply S(ρ), where ρ is the density matrix of the ensemble E [11] .. In the first stage of the above scenario, when Bob can measure only individual particles, we know that Q(ρ) is a lower bound on the information that can be conveyed per particle. As the codeword length increases to infinity, I m increases to S(ρ). One is led to speculate that for intermediate codeword lengths, R α (ρ) may play a role. For example, it is conceivable that when Alice and Bob are using codewords of length m, R α (ρ) is a lower bound on I m , where α = e −c(m−1) for some universal constant c. As m approaches infinity, then, the lower bound would approach R 0 (ρ) = S(ρ), as it should.. We can extend this idea to the study of the classical capacity of a quantum channel. At present one does not have a simple way of calculating this capacity for all channels, only because it is not known whether the amount of information conveyed can be increased by using inputs that are entangled between different uses of the channel [14] . If we disallow entangled inputs, then the resulting capacity-called the Holevo capacity-is given by a simple expression [1, 2] : it is the maximum, over all input ensembles, of the quantity S(ρ) − i p i S(ρ i ). Here {(ρ i , p i )} is the output ensemble, and ρ is its average density matrix i p i ρ i . As in the preceding paragraph, achieving this capacity requires that Bob be able to make joint measurements on arbitrarily long blocks. But suppose that Bob cannot make such measurements; suppose that he can measure only blocks of size m. For the case m = 1, it is known that the information I 1 that he can gain per particle is bounded below by max[Q(ρ) − i p i Q(ρ i )], the maximum being over all input ensembles [9] . Just as in the preceding paragraph, we can speculate that for arbitrary m, the information I m that one can convey per use of the channel is bounded below by max[R α (ρ) − i p i R α (ρ i )], with α given by α = e −c(m−1) . Of course this statement is quite speculative and we would not even want to claim it as a conjecture. We present it only to suggest how the quantity R α might conceivably be applied.. What we do have at present are a set of functions that share some mathematical properties with entropy and subentropy. There is a certain elegance in the mathematics, but whether this elegance translates into value for physics remains to be seen.

O B O O O O O B O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O


PROKARYO:

an

illustrative

and

interactive

computational

model

of

the

lactose

operon

in

the

bacterium

Escherichia

coli

Background:

We

are

creating

software

for

agent-based

simulation

and

visualization

of

bio-molecular

processes

in

bacterial

and

eukaryotic

cells.

As

a

first

example,

we

have

built

a

interactive

computer

model

of

an

Escherichia

coli

bacterium

and

its

associated

biomolecular

processes.

Our

illustrative

model

focuses

on

the

gene

regulatory

processes

that

control

the

expression

of

genes

involved

in

the

lactose

operon.

Prokaryo,

our

agent-based

cell

simulator,

incorporates

cellular

structures,

such

as

plasma

membranes

and

cytoplasm,

as

well

as

elements

of

the

molecular

machinery,

including

RNA

polymerase,

messenger

RNA,

lactose

permease,

and

ribosomes.

Results:

The

dynamics

of

cellular

'agents'

are

defined

by

their

rules

of

interaction,

implemented

as

finite

state

machines.

The

agents

are

embedded

within

a

virtual

environment

with

simulated

physical

and

electrochemical

properties.

The

hybrid

model

is

driven

by

a

combination

of

mathematical

equations

(DEQs)

to

capture

higher-scale

phenomena

and

agent-based

rules

to

implement

localized

interactions

among

a

small

number

of

molecular

elements.

Consequently,

our

model

is

able

to

capture

phenomena

across

multiple

spatial

scales,

from

changing

concentration

gradients

to

one-on-one

molecular

interactions.

We

use

the

classic

gene

regulatory

mechanism

of

the

lactose

operon

to

demonstrate

our

model's

resolution,

visual

presentation,

and

real-time

interactivity.

Our

agent-based

model

expands

on

a

sophisticated

mathematical

E.

coli

metabolism

model,

through

which

we

highlight

our

model's

scientific

validity.

Conclusion:

We

believe

that

through

illustration

and

interactive

exploratory

learning

a

model

system

like

Prokaryo

can

enhance

the

general

understanding

and

perception

of

biomolecular

processes.

Our

agent-DEQ

hybrid

modeling

approach

can

also

be

of

value

to

conceptualize,

illustrate,

and-eventually-validate

cell

experiments

in

the

wet

lab.

The

hybrid

approach

produces

results

that

are

in

good

alignment

with

the

results

from

the

continuous

Santillan

model

Comparing

mRNA

predicted

by

the

differential

equations

and

the

hybrid

model,

the

trend

of

mRNA

generation

and

degradation

is

very

similar

(Figs.

and

On

average,

permeases

are

generated

to

transport

lactose

into

the

cell

as

shown

by

both

the

differential

equations

and

the

hybrid

model

(Figs.

and

Both

models

predict

the

same

rate

of

lactose

metabolism

(in

orders

of

as

illustrated

in

Figs.

and

We

have

presented

a

hybrid

model

of

E.

coli

metabolism

related

to

the

lactose

operon.

The

dynamics

of

cellular

'agents'

are

defined

by

their

rules

of

interaction,

implemented

as

finite

state

machines.

The

agents

are

embedded

within

a

virtual

environment

with

simulated

physical

and

electrochemical

properties.

The

hybrid

model

is

driven

by

a

combination

of

mathematical

equations

(DEQs)

to

capture

higher-scale

phenomena

and

agent-based

rules

to

implement

localized

interactions

among

a

small

number

of

molecular

elements.

By

reproducing

results

from

in

silico

experiments

reported

in

the

literature,

we

have

demonstrated

that

our

hybrid

model

is

able

to

capture

phenomena

across

multiple

spatial

scales,

from

changing

concentration

gradients

to

one-on-one

molecular

interactions..

Our

article

also

demonstrates

how

interactivity

and

illustrations,

in

the

form

of

detailed

graphical

representations

in

scenes,

can

enhance

the

understanding

of

biological

systems

through

exploration

and

visualization.

We

have

implemented

our

Prokaryo

model

on

desktop

computers,

but

also

on

mobile

devices

with

touch

interfaces

(Fig.

Mobile

computing

devices

make

such

models

more

accessible.

Touch

interfaces

greatly

facilitate

navigation

through

and

exploration

of

such

models..

Prokaryo

is

only

our

first

interactive

model

of

a

larger

biomolecular

system.

We

will

extend

Prokaryo

by

including

other

pathways

(such

as

chemotaxis)

and

embed

Prokaryo

into

a

colony

of

interacting

cells.

Prokaryo

is

part

of

the

Lindsay

Virtual

Human

(LINDSAY

a

multi-scale

computational

physiology

simulation

environment

in

which

we

can

place

E.

coli

cells

inside

the

LINDSAY

intestinal

tract.

Any

cell

can

be

selected,

and

one

can

navigate

into

its

'cellular

universe'

which

is

what

we

have

described

here

as

Prokaryo.

More

information

on

the

Lindsay

Virtual

Human

can

be

found

at

including

a

video

demonstration

and

executable

download

Computational

modeling

and

analysis

provide

insights

into

complex

systems

and

can

be

used

in

education

and

research.

Two

common

approaches

for

developing

complex

system

abstractions

are

Differential

Equation

(DE)

models

and

multi-agent

(MA)

models

Both

modeling

paradigms

have

their

own

strengths

and

weaknesses.

The

main

challenge

in

building

a

bacterium

model

in

silico

is

to

capture

emergent

properties

that

of

the

constituent

parts

alone

possesses.

Two

general

modeling

approaches

are

explored

in

order

to

model

and

simulate

a

bacterium

and

to

view

the

cellular

processes

from

a

holistic

perspective.

First,

we

use

parameterized

systems

of

DEs

that

describe

the

lac

operon

dynamics

(Table

Second,

a

hybrid

agent-based

approach

models

the

lac

operon's

heterogeneous

entities

(agents)

in

an

evolving

network

of

interactions

(Figs.

and

Other

whole

cell

in-silico

simulation

systems

have

been

developed,

most

notably

Virtual

Cell

and

Smoldyn

Virtual

Cell

converts

mathematical

descriptions

of

reaction

networks

into

differential

equations

handed

to

numerical

solvers.

The

output

is

software

code

that

can

be

used

for

further

simulation

analysis.

Virtual

Cell

offers

graphical

interfaces

to

define

reactions

and

to

access

shared

simulations.

In

contrast

to

Prokaryo,

no

user

interaction

is

possible

during

the

simulations.

Smoldyn,

on

the

other

hand,

simulates

molecules

as

individual

units,

leads

to

the

folding

which

diffuse

and

react,

and

captures

natural

stochasticity

in

cell

scale

environments.

Although

various

forms

of

output

are

possible

(data

files,

image

sequences,

movies),

Smoldyn

only

offers

a

command

line

interface,

and

is

not

built

for

realtime

interaction

during

simulations.

In

contract

to

both

Virtual

Cell

and

Smoldyn,

Prokaryo's

focus

is

on

real-time

interaction

in

simulated

cell

spaces,

with

emphasis

on

illustrative

and

educational

presentation

of

its

simulations.

Prokaryo's

code

base

can

handle

Table

Mathematical

model

to

capture

counts

of

intercellular

mRNA

(M),

LacZ

(E)

and

lactose

(L)

as

reproduced

from

A

summary

of

key

symbols

is

listed

in

Table

with

constants

explained

in

Table

general

cell

simulations,

such

as

Virtual

Cell

and

Smoldyn

but

is

here

primarily

presented

as

a

simulator

to

capture

molecular

reactions

related

to

the

lactose

operon.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Deep Learning with Hierarchical Convolutional Factor Analysis Abstract-Unsupervised multilayered ("deep") models are considered for imagery. The model is represented using a hierarchical convolutional factor-analysis construction, with sparse factor loadings and scores. The computation of layer-dependent model parameters is implemented within a Bayesian setting, employing a Gibbs sampler and variational Bayesian (VB) analysis that explicitly exploit the convolutional nature of the expansion. To address large-scale and streaming data, an online version of VB is also developed. The number of dictionary elements at each layer is inferred from the data, based on a beta-Bernoulli implementation of the Indian buffet process. Example results are presented for several image-processing applications, with comparisons to related models in the literature. T HERE has been significant recent interest in multilayered or "deep" models for representation of general data, with a particular focus on imagery and audio signals. These models are typically implemented in a hierarchical manner by first learning a data representation at one scale and using the factor scores or parameters learned at that scale as inputs for the next level in the hierarchy. Methods that have been considered include deconvolutional networks [1] , convolutional networks [2] , deep belief networks (DBNs) [3] , hierarchies of sparse autoencoders [4] , [5] , [6] , and convolutional restricted Boltzmann machines (RBMs) [7] , [8] , [9] . A key aspect of many of these algorithms is the exploitation of the convolution operator, which plays an important role in addressing large-scale problems, as one must typically consider all possible shifts of canonical bases or filters. In such analyses one must learn the form of the dictionary, as well as the associated coefficients. Concerning the latter, it has been recognized that a preference for sparse coefficients is desirable [1] , [8] , [9] , [10] .. When taking the outputs of layer l and using them as inputs for layer l þ 1, researchers have investigated several processing steps that improve computational efficiency and can also improve model performance. Specifically, researchers have pooled proximate parameters from layer l, and a processing step is employed before the coefficients are employed as inputs to layer l þ 1. It has been found desirable to just keep the maximum-amplitude coefficient within each pooled region; this is termed "max-pooling" [7] , [8] . Max-pooling has two desirable effects: 1) As one moves to higher levels, the number of input coefficients diminishes, aiding computational speed; and 2) it has been found to improve classification [11] , [12] , [13] .. Some of the multilayered models have close connections to overcomplete dictionary learning [14] , in which image patches are expanded in terms of a sparse set of dictionary elements. The deconvolutional and convolutional networks in [1] , [7] , [9] similarly represent each level of the hierarchical model in terms of a sparse set of dictionary elements; however, rather than separately considering distinct patches as in [14] , the work in [1] , [7] allows all possible shifts of dictionary elements for representation of the entire image at once (not separate patches).. All of the methods discussed above, for "deep" models and for sparse dictionary learning for image patches, require one to specify a priori the number of dictionary elements employed within each layer of the model. In many applications, it may be desirable to infer the number of required dictionary elements based on the data itself. Within the deep models, for example, the dictionary-element coefficients at layer l are used (perhaps after pooling) as input features for layer l þ 1; this corresponds to a problem of inferring the proper number of features for the data of interest while allowing for all possible shifts of the dictionary elements, as in the various convolutional models discussed above. The idea of learning an appropriate number and composition of features has motivated the Indian buffet process (IBP) [15] , as well as the beta-Bernoulli process to which it is closely connected [16] , [17] . Such methods have been applied recently to (single-layer) dictionary learning in the context of image patches [18] . Further, the IBP has recently been employed for design of "deep" graphical models [19] , although the problem considered in [19] did not consider multilayer feature learning.. In this paper, we demonstrate that the idea of building an unsupervised deep model may be cast in terms of a hierarchy of factor-analysis models, with the factor scores from layer l serving as the input (potentially after pooling) to layer l þ 1. Of the various unsupervised deep models discussed above, the factor analysis view of hierarchical and unsupervised feature learning is most connected to [1] , where an ' 1 sparseness constraint is imposed within a convolution-based dictionary expansion (equivalent to a sparseness constraint on the factor scores). In this paper, we consider four differences with previous deep unsupervised models:. 1. the form of our model at each layer is different from that in [1] in that [1] performs optimization and here we employ Bayesian learning with different types of convolutional update equations; 2. the number of canonical dictionary elements or factor loadings at each layer is inferred from the data by an IBP/beta-Bernoulli construction; 3. sparseness on the dictionary-element coefficients (factor scores) at each layer is imposed with a Student-t prior (rather than ' 1 regularization); 4. fast computations are performed using Gibbs sampling and variational Bayesian (VB) analysis, where the convolution operation is exploited directly within the update equations. Furthermore, to address large-scale datasets, including those arriving in a stream, online VB inference is also developed.. While the form of the proposed model is most related to [1] , the characteristics of our results are more related to [7] . Specifically, when designing the model for particular image classes, the higher layer dictionary elements have a form that is often highly intuitive visually. To illustrate this and to also highlight unique contributions of the proposed model, consider Fig. 1 , which will be described in further detail in Section 5; these results are computed with a Gibbs sampler, and similar results are found with batch and online VB analysis. In this example, we consider images from the "car" portion of the Caltech 101 image database. In Fig. 1a , we plot inferred dictionary elements at layer 2 in the model, and in Fig. 1d we do the same for dictionary elements at layer 3 (there are a total of three layers and the layer-1 dictionary elements are simple "primitives," as discussed in detail in Section 5). All possible shifts of these dictionary elements are efficiently employed, via convolution, at the respective layer in the model. Note that at layer 2 the dictionary elements often capture localized details on cars, while at layer 3 the dictionary elements often look like complete cars ("sketches" of cars). To the authors' knowledge, only the (very distinct) model in [7] achieved such physically interpretable dictionary elements at the higher layers in the model. A unique aspect of the proposed model is that we infer a posterior distribution on the required number of dictionary elements, based on Gibbs sampling, with the numerical histograms for these numbers shown in Figs. 1c and 1f for layers 2 and 3, respectively. Finally, the Gibbs sampler infers that dictionary elements are needed for expanding each layer, and in Figs. 1b and 1e the use of dictionary elements is shown for one Gibbs collection sample, highlighting the sparse expansion in terms of a small subset of the potential set of dictionary elements. These results give a sense of the characteristics of the proposed model, which is discussed in detail in the subsequent discussion.. The remainder of the paper is organized as follows: In Section 2, we develop the multilayer factor analysis viewpoint of deep models. The detailed form of the Bayesian model is discussed in Section 3, and methods for Gibbs, VB, and online VB analysis are discussed in Section 4. A particular focus is placed on explaining how the convolution operation is exploited in the update equations of these iterative computational methods. Several example results are presented in Section 5, with conclusions provided in Section 6. An early version of the work presented here was published in a conference paper [20] which focused on a distinct hierarchical beta process construction, and in which the batch and online VB computational methods were not developed. The development of deep unsupervised models has been cast in the form of hierarchical factor analysis, where the factor loadings are sparse and characterized by a unique convolutional form. The factor scores from layer l serve as the inputs to layer l þ 1, with a max-pooling step. By framing the problem in this manner, we can leverage significant previous research with factor analysis [41] . Specifically, a truncated beta-Bernoulli process [16] , [17] , motivated by the IBP [15] , has been used on the number of dictionary elements needed at each layer in the hierarchy (with a particular focus on inferring the number of dictionary elements at higher levels, while at layer 1 we typically set the model with a relatively small number of primitive dictionary elements). We also employ a hierarchical construction of the Student-t distribution [23] to impose sparseness on the factor scores, with this related to previous sparseness constraints imposed via ' 1 regularization [1] .. The inferred dictionary elements, particularly at higher levels, typically have descriptive characteristics when viewed in the image plane.. Summarizing observations, when the model was employed on a specific class of images (e.g., cars, planes, seats, motorcycles, etc.), the Layer-3 dictionary elements when viewed in the image plane looked very similar to the associated class of images (see Fig. 8 ), with similar (but less fully formed) structure seen for Layer-2 dictionary elements. Further, for such single-class studies the betaBernoulli construction typically only used a relatively small subset of the total possible number of dictionary elements at Layers 2 and 3, as defined by the truncation level. However, when considering a wide class of "natural" images at once, while the higher level dictionary elements were descriptive of localized structure within imagery, they looked less like any specific entity (see Fig. 12 ); this is to be expected by the increased heterogenity of the imagery under analysis. Additionally, for the dictionary-element truncation considered, as the diversity in the images increased it was more likely that all of the possible dictionary elements within the truncation level were used by at least one image. Nevertheless, as expected from the IBP, there was a relatively small subset of dictionary elements at each level that were "popular", or widely used (see Section 3.1).. In general, we find that the posterior distribution on the number of dictionary elements may change as a function of the type of images considered. However, based upon running the IBP framework on a given class of images, one can safely set the number of dictionary elements at the mode of the posterior. For similar images in the future one may use that setting and avoid the IBP. We would argue that the IBP provides a "principled means of model exploration. One could alternatively explore (with perhaps exhaustive testing) different settings for the number of dictionary elements, but the proposed approach is more systematic, and perhaps valuable when new classes of images are considered. Guillermo Sapiro received the PhD degree in electrical engineering from the Technion in Israel. He is a professor in the Electrical and Computer Engineering Department at Duke University. He is a senior member of the IEEE.. David Blei received the PhD degree in electrical engineering from the University of California, Berkeley. He is an associate professor in the Computer Science Department at Princeton University. He is a member of the IEEE.. David Dunson received the PhD degree in statistics from Emory University. He is a professor in the Statistics Department at Duke University.. Lawrence Carin received the PhD degree in electrical engineering from the University of Maryland. He is a professor in the Electrical and Computer Engineering Department at Duke University. He is a fellow of the IEEE and a member of the IEEE Computer Society.. . For more information on this or any other computing topic, please visit our Digital Library at www.computer.org/publications/dlib.

B I O O B B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O