Archetype relational mapping - a practical openEHR persistence solution Background: One of the primary obstacles to the widespread adoption of openEHR methodology is the lack of practical persistence solutions for future-proof electronic health record (EHR) systems as described by the openEHR specifications. This paper presents an archetype relational mapping (ARM) persistence solution for the archetype-based EHR systems to support healthcare delivery in the clinical environment. Methods: First, the data requirements of the EHR systems are analysed and organized into archetype-friendly concepts. The Clinical Knowledge Manager (CKM) is queried for matching archetypes; when necessary, new archetypes are developed to reflect concepts that are not encompassed by existing archetypes. Next, a template is designed for each archetype to apply constraints related to the local EHR context. Finally, a set of rules is designed to map the archetypes to data tables and provide data persistence based on the relational database. Results: A comparison study was conducted to investigate the differences among the conventional database of an EHR system from a tertiary Class A hospital in China, the generated ARM database, and the Node + Path database. Five data-retrieving tests were designed based on clinical workflow to retrieve exams and laboratory tests. Additionally, two patient-searching tests were designed to identify patients who satisfy certain criteria. The ARM database achieved better performance than the conventional database in three of the five data-retrieving tests, but was less efficient in the remaining two tests. The time difference of query executions conducted by the ARM database and the conventional database is less than 130 %. The ARM database was approximately 6-50 times more efficient than the conventional database in the patient-searching tests, while the Node + Path database requires far more time than the other two databases to execute both the data-retrieving and the patient-searching tests. Conclusions: The ARM approach is capable of generating relational databases using archetypes and templates for archetype-based EHR systems, thus successfully adapting to changes in data requirements. ARM performance is similar to that of conventionally-designed EHR systems, and can be applied in a practical clinical environment. System components such as ARM can greatly facilitate the adoption of openEHR architecture within EHR systems. This paper presents an ARM persistence solution for archetype-based EHR systems. ARM uses archetypes to generate relational databases and achieve similar performance as compared to conventional databases in data-retrieving queries. ARM takes great advantage of the CKM public archetype repository to facilitate data manipulation with well-defined archetypes for clinicians and to achieve better performance in patient-searching queries. System components like ARM can facilitate the adoption of open-EHR architecture in EHR systems. The authors will continue to complete the mapping rules according to the semantics of archetypes, improve ARM constraints and the performance of the generated databases, design and implement data access services, and perform thorough tests of the ARM approach in real clinical environments. This paper presents an ARM persistence solution for archetype-based EHR systems. While the ARM approach is designed to generate a relational database from archetypes and templates and can achieve performance similar to a conventionally-designed database, there were several encountered challenges and issues. Fig. 9 Node + Path database schema. In all figures, PK stands for primary key. FK stands for foreign key, and indicates the data item which current data item relates to. SLOT indicates target archetype which conform to current data item. Solid line indicates foreign key relationship between archetypes. Dash line indicates composition relationship through slot between archetypes. IDI stands for identification data item. QDI stands for query data item. CI stands for clustered indexed. NCI stands for non-clustered indexed. Data items in italic type are not covered by archetypes ARM deployment ARM employs a model-driven approach to allow data persistence to adapt to changes in data requirements according to archetypes that represent general domain concepts and templates tailored to ARM constraints. The mapping rules are implemented in a persistent service to automatically generate the database, and avoid necessary manual uploading of the database. Currently, the changes described in archetype semantic relationships are easy to implement on the relational database, but one change is not explicitly included in the new versions of the archetypes. In ideal archetypes, the data type of data items should be as stable as the data items, and remain unchanged. However, this cannot be avoided during archetype development, particularly for archetypes that are initially developed from local data requirements and later extended to a global scope, which often results in incompatible versions of archetypes.. Changing the column data type can induce chaos into the relational database; thus, two mapping rules are designed to automatically adapt to change. One rule maps each version of an archetype to a table; old versions of an archetype will gradually become outdated and obsolete, and can then be safely moved to a backup database. Prior to removal, multiple versions of an archetype that are simultaneously in service result in a large number of tables . The second rule maps all versions of an archetype to a new table, then imports all data from the old table into the new one according to the conversion algorithm provided with the archetype [30] ; the old table is then removed. However, the data conversion process is costly in terms of time and computation if the table contains a great number of records. Although it is safe to redeploy changed archetypes and templates to update a deployed database, the principles of archetype design recommend that archetypes and templates be maintained by a committee of domain experts, and deployed when they are stable. Archetypes should be reused as much as possible to represent the domain concepts. Templates are used to align the data persistence to different data requirements, and to avoid shifting the heterogeneity of data requirements to archetypes. Currently, an electronic health record (EHR) system is essential to support clinical practice with information technology in healthcare environments. EHR is a repository of information regarding the health status of a subject of care in computer processable form [1] . However, healthcare data is generally too complicated, flexible, and changeable to capture a universal, comprehensive and stable schema of information, which is the foundation of the entire EHR architecture. Highly-specialized and complex EHR systems cannot acclimate to the evolution of healthcare data requirements, which require EHR systems to embrace a dynamic, state-of-the-art, rapidly evolving information infrastructure [2] . In order to protect EHR systems from changes in the healthcare domain, openEHR [3] has published a series of specifications to guide the development of future-proof EHR systems. Solutions span from information modelling to system architecture, to meet the continually evolving needs of EHR systems.. The concept of openEHR focuses on systems and tools necessary to the computation of complex and constantly evolving health information at a semantic level, according to the following three paradigms: separation of information models, domain content models, and terminologies; separation of responsibilities; and separation of viewpoints [4] . Among these three paradigms, the separation of information models, domain content models and terminologies promotes a significant shift from the single-level modelling approach of information system development to a twolevel modelling approach. In the single-level modelling approach, domain concepts that are processed by the EHR system are hard-coded directly into the application and database models. In two-level modelling, the semantics of information and knowledge are separated into a small, comprehensible, non-volatile reference model (RM), which is used to build information systems and knowledge models; archetypes are used as formalisms and structures to express numerous and volatile domain concepts [5] .. The RM represents the general features of health record components, their method of organization, and necessary contextual information to satisfy both the ethical and legal requirements of the health record. The RM encompasses the stable features of the health record by defining the set of classes that composes the blocks which in turn constitute the record. Archetypes define entire, coherent informational concepts from the clinical domain [6] . An archetype is a hierarchical combination of components from the RM with available restrictions placed on names, possible data types, default values, cardinality, etc. These structures, although sufficiently stable, may be modified or replaced by others as clinical practice progresses and evolves [7] . Archetypes are deployed at runtime via templates that specify particular groups of archetypes to be used for a particular purpose, often corresponding to a screen form. A template is a specification that creates a tree structure of one or more archetypes, and each constraining instance of various RM types such as composition, section, entry subtypes, etc. Templates typically closely correspond to screen forms, printed reports, and generally complete information to be captured or sent at the application level; they may therefore be used to define message content [4] .. By utilizing a two-level modelling approach, EHR systems can be built on a stable RM as a general framework, and thus use archetypes as the domain information model to achieve greater flexibility and stability, particularly in situations in which the domain concepts are vast in number, have complex relationships, and evolve continuously. The responsibilities of specialists from the information technology domain and the healthcare domain are disengaged: developers focus only on the technical components of EHR systems, while specialists develop the structural model based on domain concepts and archetypes. This enables domain specialists to participate directly in the production of the artefacts that are interpreted by EHR systems, to organize and present healthcare information, and to control the EHR system without intervention from the system supplier or re-programming.. Syntactic and semantic interoperability between different EHR systems is facilitated by the use of archetypes. However, agreement between data content and the information models is necessary for real integration and semantic interoperability [8] . In the single-level modelling approach, the domain concepts are implicitly contained within the EHR systems. No consensus healthcare domain models exist in healthcare domain models to which EHR systems may conform; each system may model different aspects and granularity levels of domain concepts, resulting in heterogeneity. In the two-level modelling approach, the RM ensures that EHR systems can always send information to other systems and receive readable information in return, thus ensuring data interoperability. Archetypes can be used as the common knowledge repository to share evolving clinical information that can be processed by the receiving systems, thus enabling semantic interoperability [9] . The openEHR system maintains Clinical Knowledge Manager (CKM) as an official archetype repository to support the governance of international domain knowledge.. Specifications of openEHR can be used to create and sustain a flexible EHR ecosystem that consists of may integrated services, which is too complex to be accurately processed by single applications [10] . The general architectural approach can be considered as five layers: persistence (data storage and retrieval), back-end services (including EHR, demographics, terminology, archetypes, security, record location, etc.), virtual EHR (a coherent set of APIs to the various back-end services and an archetype-andtemplate-enabled kernel responsible for creating and processing archetype-enabled data), application logic (user applications or another service, such as a query engine), and presentation (the graphical interface of the application) [4] . The archetypes share the openEHR innovation of adaptability because they are external to the software, while key components of the software are derived from the archetypes [11] . Much current research has been devoted to the use of archetypes to drive the persistence, accessibility, and presentation of healthcare information systems [12] [13] [14] [15] [16] .. Most EHR systems in the healthcare field are built according to the single-level modelling approach, despite the many advantages of two-level modelling. One of the primary reasons is that the persistence layer is inadequate to meet the requirements of clinical practice. As the foundation of EHR systems, the persistence layer determines the EHR system architecture, and thus also function as a performance bottleneck. The openEHR system promotes a Node + Path persistence solution that serializes sub-trees of fine-grained data into blobs or strings based on the object or relational systems [17] . The data can be serialized according to different granularity levels, from top-level information objects to the lowest leaf nodes. In essence, the Node + Path solution is an entity-attribute-value (EAV) approach, which takes advantage of the semantic paths in openEHR data to improve the serialized-blob design. The greatest advantages of the Node + Path approach are flexibility and simplicity; all data nodes are serialized, and their paths are recorded adjacently in a two-column table of < node path, serialized node value>. However, the simplicity of the data storage structure induces complex data retrieval logic, which strains the performance of data insertion and query, which requires flexibility [18] . Some researchers have reported similar performances in the processing of entity-centered queries by conventional and EAV models, but that EAV models were approximately 3-5 times less efficient in the processing of attribute-centered queries [19] . Evaluations of persistence solutions using an XML database also indicates that XML databases were considerably slower and required much more space than the relational database [20] . There has also been similar research into the data persistence of alternative approaches based on two-level modelling, such as EN13606 [21] or HL7 Version 3 [22] . In a proof-of-concept work of EN13606 [7] , the data storage was developed by applying Object Relational Mapping (ORM) [23] to the RM of EN13606; this approach was investigated by the authors in earlier work [24] . The deep inheritance and complicated relationships of the EN13606 RM induces excessive JOIN operation during data query. Additionally, classes near the top hierarchy become heavily overloaded with data. For example, DATA_VALUE is the basic class of all data types, and contains the common attributes of all instances of all data classes, but it is unable to operate in real time [7] . IBM Clinical Genomics medical research developed a hybrid data model based on the HL7 Version 3 Reference Information Model (RIM) [25] . The hybrid data model combines elements of both the ORM and EAV approaches, which is well-suited to the sparse and flexible data of a medical research data warehouse. However, it demonstrates similar problems to other ORM and EAV approaches in that it does not improve the performance enough to support effective clinical transactions.. The relational database is still of primary importance to the data persistence of EHR systems, and its excellent performance has already been well proved in many successful EHR projects and widely-accepted EHR productions. The primary challenge to the application of a two-level modelling approach to a relational database is that the domain information model is hard-wired into the database model; when the domain information model changes, the relational database must be redesigned in order to facilitate the new domain information model. There are two approaches to support the adaptation of the persistence layer to changes in archetypes. One approach is to use a general data storage structure that is independent of archetypes, such as Node + Path, which has been well-investigated. The other approach is to generate a database model to design a persistence layer driven by archetypes.. This work presents an archetype relational mapping (ARM) persistence solution based on a relational database that can achieve similar performance to the conventional database in practical clinical environments. This work extends the basic ARM method introduced in a previous proof-of-concept work [24] by providing a more sophisticated mapping approach based on templates and archetypes in order to map archetypes to relational tables. Performance optimization rules and details are then provided. First, the ARM approach is introduced in detail, including archetype modelling, template definition, and mapping rules. Then, the ARM is applied to the EHR data requirements of a tertiary hospital in China. A comparison of the generated ARM database, the conventional database deployed in the hospital, and the openEHR official Node + Path database is conducted. After analysis, the challenges encountered during ARM development are discussed, and conclusions are provided. An underpinning principle of openEHR is the use of archetypes and templates, which represent formal models of domain content that are used to control data structure and content during creation, modification, and querying [26] . The ARM approach employs archetypes and templates to generate a persistence model and provide data access based on a relational database. The ARM is intended to fulfill several functionalities:

B I I O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O


Confidence-

and

margin-based

MMI/MPE

discriminative

training

for

off-line

handwriting

recognition

We

present

a

novel

confidence-and

margin-based

discriminative

training

approach

for

model

adaptation

of

a

hidden

Markov

model

(HMM)-based

handwriting

recognition

system

to

handle

different

handwriting

styles

and

their

variations.

Most

current

approaches

are

maximumlikelihood

(ML)

trained

HMM

systems

and

try

to

adapt

their

models

to

different

writing

styles

using

writer

adaptive

training,

unsupervised

clustering,

or

additional

writerspecific

data.

Here,

discriminative

training

based

on

the

maximum

mutual

information

(MMI)

and

minimum

phone

error

(MPE)

criteria

are

used

to

train

writer-independent

handwriting

models.

For

model

adaptation

during

decoding,

an

unsupervised

confidence-based

discriminative

training

on

a

word

and

frame

level

within

a

two-pass

decoding

process

is

proposed.

The

proposed

methods

are

evaluated

for

closed-vocabulary

isolated

handwritten

word

recognition

on

the

IFN/ENIT

Arabic

handwriting

database,

where

the

word

error

rate

is

decreased

by

relative

compared

to

a

ML

trained

baseline

system.

On

the

large-vocabulary

line

recognition

task

of

the

IAM

English

handwriting

database,

the

word

error

rate

is

decreased

by

relative.

Most

state-of-the-art

single-pass

and

multi-pass

HMM-based

handwriting

recognition

systems

are

trained

using

the

maximum-likelihood

(ML)

criterion..

Typical

training

criteria

for

string

recognition

like,

for

example,

minimum

phone

error

(MPE)

and

maximum

mutual

information

(MMI)

in

speech

recognition

are

based

on

a

(regularized)

loss

function.

In

contrast,

large

margin

classifiersthe

de-facto

standard

in

machine

learning-maximize

the

separation

margin.

An

additional

loss

term

penalizes

misclassified

samples..

The

MMI

training

criterion

has

been

used

in

to

improve

the

performance

of

an

HMM-based

off-line

Thai

handwriting

recognition

system

for

isolated

characters.

They

propose

a

feature

extraction

based

on

a

block-based

PCA

and

composite

image

features,

which

are

reported

to

better

at

discriminating

Thai

confusable

characters.

In

the

authors

apply

the

Minimum

Classification

Error

(MCE)

criterion

to

the

problem

of

recognizing

online

unconstrained-style

characters

and

words

and

report

large

improvements

on

a

writerindependent

character

recognition

task

when

compared

to

a

ML

trained

baseline

system.

Similar

to

the

system

presented

in

we

apply

the

MMI

criterion,

but

modified

by

a

margin

term.

This

margin

term

can

be

interpreted

as

an

additional

observation-dependent

prior

weakening

the

true

prior

and

is

identical

with

the

SVM

optimization

problem

of

log-linear

models

The

most

common

way

for

unsupervised

adaptation

is

the

use

of

the

automatic

transcription

of

a

previous

recognition

pass

without

the

application

of

confidence

scores.

Many

publications

in

automatic

speech

recognition

(ASR)

have

shown

that

the

application

of

confidence

scores

for

adaptation

can

improve

recognition

results.

However,

only

small

improvements

are

reported

for

confidence-based

maximum-likelihood

linear

regression

(MLLR)

adaptation

or

constrained-MLLR

adaptation

In

this

work,

we

present

a

novel

unsupervised

confidence-based

discriminative

model

adaptation

approach.

This

paper

briefly

reviews

how

the

MMI/MPE

training

criteria

can

be

extended

to

incorporate

the

margin

concept,

and

that

such

modified

training

criteria

are

smooth

approximations

to

support

vector

machines

with

the

respective

loss

function

In

addition

to

the

margin

concept,

the

MMI/MPE

training

criteria

are

extended

by

an

additional

confidence

term

to

allow

for

novel

unsupervised

model

adaptation..

The

focus

of

this

work

shall

be

on

off-line

handwriting

recognition

of

closed-vocabulary

isolated

words

and

largevocabulary

sentence

recognition

tasks

in

combination

with

m-gram

language

models.

More

explicitly,

the

novelties

of

our

investigation

are

as

follows:.

Direct

evaluation

of

the

utility

of

the

margin

term

in

MMI/MPE-based

training.

Ideally,

we

can

turn

on/off

the

margin

term

in

the

optimization

problem.

Direct

evaluation

of

the

utility

of

an

additional

confidence

term.

Ideally,

we

improve

over

the

best

trained

system

by

retraining

the

system

with

unsupervised

labeled

test

data.

Direct

evaluation

of

the

amount

of

iterations

and

confidence

thresholds

during

optimization.

In

ASR,

typically

a

low

number

of

iterations

is

used

in

optimization

and

confidence

thresholds

are

optimized

on

small

subsets

only.

Here,

we

allow

for

a

high

number

of

iterations

on

large

data

sets,

and

a

detailed

analysis

of

confidences

in

unsupervised

model

adaptation.

Evaluation

on

state-of-the-art

systems.

Ideally,

we

directly

improve

over

the

best

discriminative

system,

e.g.,

conventional

(i.e.,

without

margin)

MMI/MPE

for

handwriting

recognition..

Due

to

the

nature

of

the

novel

publicly

available

RWTH

OCR

framework

and

databases,

it

can

be

assumed

that

most

results

can

be

transferred

to

ASR

domains.

Similar

usage

of

features,

lexica,

and

language

models

on

smaller

corpora

allow

for

a

detailed

analysis

of

regularization,

optimization

iterations,

as

well

as

impact

of

confidence

thresholds..

The

proposed

approach

takes

advantage

of

the

generalization

bounds

of

large

margin

classifiers

while

keeping

the

efficient

framework

for

conventional

discriminative

training.

This

allows

us

to

directly

evaluate

the

utility

of

the

margin

term

for

handwriting

recognition.

So,

our

approach

combines

the

advantages

of

conventional

training

criteria

and

of

large

margin

classifiers.

We

presented

a

novel

confidence-and

margin-based

discriminative

training

using

a

MMI/MPE

training

criterion

for

model

adaptation

in

off-line

handwriting

recognition.

The

advantages

of

the

proposed

methods

using

an

HMM-based

multi-pass

decoding

system

were

shown

for

Arabic

handwriting

on

the

IFN/ENIT

corpus

(isolated

word

recognition)

and

for

Latin

handwriting

on

the

IAM

corpus

(large-vocabulary,

continuous

sentence

recognition).

Both

approaches

showed

their

robustness

w.r.t.

transcription

errors

and

outperformed

the

maximum-likelihood

(ML)

trained

baseline

models.

We

discussed

an

approach

how

to

modify

existing

training

criteria

for

handwriting

recognition

like,

for

example,

MMI

and

MPE

to

include

a

margin

term.

The

modified

training

criterion

M-MMI

was

shown

to

be

closely

related

to

existing

large

margin

classifiers

(e.g.,

SVMs)

with

the

respective

loss

function.

This

approach

allows

for

the

direct

evaluation

of

the

utility

of

the

margin

term

for

handwriting

recognition.

As

expected,

the

benefit

from

the

additional

margin

term

clearly

depends

on

the

training

conditions.

The

proposed

discriminative

training

approach

could

outperform

the

ML

trained

system

on

all

tasks..

The

impact

of

different

writing

styles

was

dealt

with

a

novel

confidence-based

discriminative

training

for

model

adaptation,

where

the

usage

of

state-confidences

during

the

iterative

optimization

process

based

on

the

modified

M-MMI-conf

criterion

could

decrease

the

word

error

rate

on

the

IFN/ENIT

database

by

relative

in

comparison

with

a

ML

trained

system.

On

the

IAM

database,

similar

improvements

could

be

observed

for

the

proposed

M-MMI-conf

and

M-MPE-conf

criteria,

leading

to

a

WER

decrease

by

relative

in

comparison

with

a

maximum-likelihood

trained

system,

and

representing

one

of

the

best

known

WER

in

the

literature

for

a

single

and

purely

HMM-based

system.

In

supervised

training,

the

M-MPE

criterion

could

outperform

the

M-MMI

approach,

whereas

in

unsupervised

and

confidence-based

model

adaptation,

the

M-MMI-conf

approach

could

clear

the

initial

gap

to

the

M-MPE

trained

model..

Interesting

for

further

research

will

remain

hybrid

HMM/

ANN

approaches

combining

the

advantages

of

large

and

non-linear

context

modeling

via

neural

networks

while

profiting

from

the

Markovian

sequence

modeling.

This

is

also

supported

by

the

relative

improvement

we

could

achieve

in

the

ICFHR

Arabic

handwriting

competition

with

the

proposed

framework

but

an

MLP-based

feature

extraction.

O O O O B I O O B I O O O O O O B I O O B I O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O B I I O O B I I O O O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I I O O B I I O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B I O O O O O O O O O O B O O B I O O O O O O O O O O O O O B O O O O O O O O O B I O O O O O O B I O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O


Similarity-Guided Streamline Placement with Error Evaluation Abstract-Most streamline generation algorithms either provide a particular density of streamlines across the domain or explicitly detect features, such as critical points, and follow customized rules to emphasize those features. However, the former generally includes many redundant streamlines, and the latter requires Boolean decisions on which points are features (and may thus suffer from robustness problems for real-world data).We take a new approach to adaptive streamline placement for steady vector fields in 2D and 3D. We define a metric for local similarity among streamlines and use this metric to grow streamlines from a dense set of candidate seed points. The metric considers not only Euclidean distance, but also a simple statistical measure of shape and directional similarity. Without explicit feature detection, our method produces streamlines that naturally accentuate regions of geometric interest.In conjunction with this method, we also propose a quantitative error metric for evaluating a streamline representation based on how well it preserves the information from the original vector field. This error metric reconstructs a vector field from points on the streamline representation and computes a difference of the reconstruction from the original vector field. Vector fields are commonly used to represent physical properties such as particle velocity or magnetic field across some domain. Visualization of vector fields is important for performing qualitative analysis in areas such as astronomy, aeronautics, meteorology, and medicine. The most common approaches are based on either line-integral convolution, which uses dense, highly local integration, or streamlines, which are longer integral curves with a more discrete, global flavor.. One particular area of scientific interest to us is the study of the turbulent behavior of ionized gases around a black hole. Velocity and magnetic field data are acquired using MHD (magnetohydrodynamics) simulation on a spherical domain. Because the domain of interest is a general 3D volume and is not restricted to a surface, sparse streamline approaches are preferable to textured approaches such as line-integral convolution.. As a sparse representation of a vector field, the quality of a set of streamlines is highly dependent on their placement, which includes a seed location and a length for each streamline.. If a specific type of feature of interest is known for the given application area and is mathematically well-defined, a feature-guided algorithm can tailor the placement to accentuate these features of interest. However, the discrete classification of features may suffer from robustness problems in practice.. When a feature-based approach is not applicable, a density-guided (or distance-guided) approach is typically employed. Density-guided approaches place streamlines to enforce a user-specified density function across the domain. The function may be constant, producing roughly evenly spaced streamlines, or it may be spatially varying (e.g., it may be mapped to vector magnitude or some other interesting scalar function). Notice that in these techniques, the density of streamlines is independent of the underlying flow function.. In this paper, we propose a similarity-guided placement strategy. We define a similarity distance as a metric for computation of local distance between streamlines. The similarity distance accounts not only for the closest distance from a point on one streamline to another streamline, but also accounts for their similarity of shape and direc-. • Yuan Chen is at Johns Hopkins University, E-mail: cheny@cs.jhu.edu.. • Jonathan D. Cohen is at Lawrence Livermore National Laboratory, E-mail: jcohen@llnl.gov.. • Julian Krolik is at Johns Hopkins University, E-mail: jhk@pha.jhu.edu. tion. This is accomplished by examining the statistics of the Euclidean distance function as measured over a spatial window.. We employ this new similarity distance metric in the context of a simple, greedy algorithm for streamline generation. A large number of seed points are randomly ordered in a processing queue. Each seed point is grown using Runge-Kutta integration until its similarity distance from any streamline would fall below a prespecified similarity tolerance. This process may be performed in multiple passes to promote streamline fairness and to generate a multi-resolution streamline representation. As in some other density-guided approaches, the similarity tolerances may either be constant across the domain or spatially varying. We show that by using the similarity distance as a metric, even this simple placement strategy produces streamlines that naturally accentuate regions of geometric interest, with sparser streamlines in regions of more parallel flow.. In addition to visually demonstrating how the similarity-guided placement algorithm behaves on simple and complex vector fields in two and three dimensions, we introduce an error metric for streamline representations. This error metric is based on the idea that the streamlines are an alternative representation for the vector field, and should thus strive to preserve the information contained in the vector field. Our approach is to reconstruct a vector field from points on the streamlines and compute a difference of the reconstructed vector field from the original. We show that the similarity-guided placement strategy produces a significantly lower error for a given number of streamlines than a pure distance-guided strategy. We have proposed the idea of a similarity metric for measuring streamline proximity, and demonstrated a streamline placement algorithm using similarity to drive the selection of streamlines and their lengths. This new approach favors streamlines near interesting flow features, such as critical points and separations, without the need to explicitly enumerate these features.. We have also introduced an error metric for streamline representations to measure how well they preserve the information in the original vector field. Using the metric, we can see that similarity-guided streamline placement outperforms Euclidean distance-guided placement not only visually, but quantitatively as well.. The placement algorithm still has some room for improvement. It may benefit from more carefully chosen order of seed point growth, such as the farthest point formulation of [10] . It may also be beneficial to add more types of moves to our optimization process, as in [13] , making the initial placement order less crucial and less sensitive to a particular randomization. A more efficient spatial search structure can be used to improve the speed of generating the streamlines.. We choose to use a very intuitive and simple reconstruction method for vector field reconstruction in our error measurement, which consists of Delaunay triangulation and linear interpolation. It would be interesting to see some other reconstruction methods, which may be more suitable for this application. So far, we have shown results on fairly simple, synthetic data as a way of validating our approach. Now we demonstrate on two more complicated data sets from real scientific simulations, then report some data from all the models. Figure 14 shows a 2D turbulent flow from a simulation of a swirling jet. It simulates the situation with an inflow into a steady medium. The domain is a structured grid with 251x159 cells. The topology of the flow has a very complicated structure and contains about 300 critical points. The extremely dense distribution of critical points makes it hard to seed streamlines using feature-guided strategies. Figure 14(a) shows the result using the typical Euclidean distance placement without detecting self closure. It contains some obvious visual clutter artifacts. The computational time is roughly three times longer than the others in Figure 14 because those closed streamlines will only stop integration when the integration length is beyond some user-defined maximum length, consuming unnecessary computational time. In Figure 14(b) , we still use the Euclidean distance, but terminate the integration of closed curves. Figure 14(c) shows the result from our similarity-guided strategy with self closure detection. The separation distance is reduced from (a) and (b) so that we get the same number of streamlines in all three. Figure 14( Figure 14 (c), so it uses fewer streamlines overall, and captures even less structure than (b) or (c). Figure 13 shows a 3D magnetic field from an MHD simulation of ionized gases around a black hole. The field is symmetric, so the streamlines may be traced through a spherical volume, though the data is in one sphere quadrant (minus some small cutouts at the center and down the polar axis). The curvilinear grid has a huge range of cell sizes, with the smallest ones at the smallest radii and closest to the equatorial plane. The magnetic field has no critical points. In Figure 13(a) , we use a variable density function to emphasize the region close to the polar axis. Streamlines in this region tend to follow a fairly tight coil up the axis. In Figure 13(b) , we emphasize the streamlines near the equatorial plane, which tend to wind around in a much broader pattern.. We have implemented the streamline generation algorithm on a Linux PC equipped with 2.4 GHz dual AMD Opteron 250 CPU, 8GB RAM. Table 1 presents some timing data for several test models. We use one seed point per grid sample. The 1.vec data ( Figure 10 ) is a simple 2D synthetic data set provided with Abdelkrim Mebarki's streamline placement package. The om08 (Figure 14 is a complex simulation of a swirling jet with an inflow into a steady medium. The 3d.vec ( Figure 4 ) is a synthetic data set we generated to contain six critical points. KdPhrg ( Figure 13 ) is a magnetic field resulting from a relativistic MHD simulation of ionized gas in proximity to a black hole. Figure 12 demonstrates the relation between the processing time and the number of streamlines generated. Since the similarity distance check is the most crucial operation in our algorithm, the computing time grows fairly linearly with the number of generated streamlines. There is also a small factor corresponding to the number of seed points. Table 2 shows the reconstruction errors of our method and Euclidean distance based method on 2D and 3D data sets. Flow visualization has a rich literature, with a number of surveys available [4, 7] . We consider here the most relevant work in streamline visualization.. There are two competing goals in streamline visualization. One is to represent interesting features as well as possible. In principle, if we are allowed to place streamlines densely enough, we will not miss any information of the field. But practically, dense representations can cause severe clutter problems, especially in 3D. Thus the second goal is to have as little clutter as possible. Both feature-guided and densityguided placements try to find a good balance between these two goals.. Feature-guided strategies begin by identifying particular features in the field, that are of interest for a certain research or engineering problem, such as boundary layers, separation lines, separation bubbles, critical points, shock waves and so on. For many applications, the vicinity of critical points contains an interesting flow pattern. A critical point is defined as a point in the vector field where all components of the vector are zeros and the streamline slope is indeterminate. So no streamline will pass through it. For example, in a velocity field, critical points are those with zero velocity magnitude. Critical points are further categorized into different types. The field is then segmented into regions, each containing a single critical point. Different seeding patterns may be used around the vicinity of the different critical points. Finally, some additional seed points are randomly distributed around the field.. Verma et al. [14] first proposed a feature-guided placement strategy for 2D vector field visualization. Ye et al. [15] extended this idea to 3D vector fields. They also provided a continuous mapping between seeding patterns and critical points, which accommodates the variability of critical points in 3D. In some applications, the number of critical points is small, so the number of streamlines generated by feature-guided strategies is likewise relatively small, and the clutter problem is minimal. But with a large number of critical points, the feature-guided approach won't necessarily produce a sparse streamline representation. Another drawback is the requirement of feature identification, which is not always available or easy.. Density-based strategies are usually associated with some density or distance metric. Turk et.al [13] proposed an image-guided placement strategy in 2D, which defined the density as the gray level of the low-pass filtered version of the image. Their optimization process adjusts the streamline placement to achieve a desired density by iteratively adding, growing, shrinking, merging, or removing streamlines.. Jobard and Lefer [5] proposed an evenly-spaced streamline placement algorithm in 2D where the density is defined by the Euclidean distance between adjacent streamlines. Mebarki et al. [10] proposed a farthest point seeding strategy for 2D vector fields. This greedy algorithm places one streamline at a time, always at the farthest point away from all existing streamlines to optimize continuity (i.e., it promotes long streamlines). They also used the Euclidean distance between adjacent streamlines as their density metric. Mattausch et al. [9] extended the idea of evenly spaced placement into 3D and used the 3D Euclidean distance as the metric. Liu et. al [16] introduced double queues for prioritizing topological seeding and favoring long streamlines. They also use cubic Hermite polynomial interpolation with RK4-ASSEC (fourth order Runge-Kutta integrator with adaptive step size and error control) to accelerate placement generation.. Li et al. [8] proposed a 3D image-space streamline placement method. They control the seeding and generation of streamlines in image space to avoid visual cluttering. This approach guarantees the minimum spacing between adjacent streamlines on the image plane and solves the clutter problem directly. But the quality of placement is highly dependent on a precomputed depth map, which provides the depth of 2D seeds in 3D object space. Schlemmer et al. [12] defined the streamline density of a region as the ratio between the number of occupied pixels by streamlines and the total number of pixels in the region. To test the use of similarity as a metric for placing streamlines, we have developed a simple, greedy algorithm for streamline growth from seed points. The pseudocode for our basic placement algorithm is shown in Figure 5 . Placement begins by inserting a densely-sampled set of seed points into the Seeds queue in a random order. For each point in the queue, we iteratively grow the streamline by applying adaptive fourth order Runge-Kutta integration. As we grow the streamline, we check its similarity distance to all the previously placed streamlines. When the similarity distance falls below the user specified threshold, d Sep , or if we reach a boundary of the volume or if integration becomes undefined due to a nearby critical point, we terminate the growth of that streamline. In the case of self separation (comparing a streamline to itself), we generally use a significantly smaller separation distance, d Sel f Sep , to allow for the case of streamlines that form a closed loop. If the resulting streamline is longer than the desired minimum length, it is added to Lines, the set of placed streamlines. Otherwise, it is discarded.. In our implementation, we accelerate the above algorithm by placing all the sample points of the streamlines into a uniform grid, where the cell size is proportional to the separation distance. Because the similarity distance between point p on streamline s i and point q on streamline s j is always greater than or equal to the Euclidean distance between p and q, there is no need to find the closest point on streamlines that are farther from p than the separation distance, nor do we need to compute the actual similarity distance in that case. Of course, more sophisticated space partitioning structures may be used in place of the uniform grid, which has been sufficient for our test examples.. Although this simple placement algorithm seems to work reasonably well, it does have some shortcomings. It is unfair to the streamlines, favoring those with seeds early in the random queue over those later in the queue. The earlier seeds have the opportunity to grow longer, although there may be nothing particularly special about them. One could order the seeds according to some importance function, but often one cannot predict the importance of a streamline simply from its seed position.. One way to mitigate this situation is to grow the streamlines in several passes, with a schedule of decreasing separation distances. This is more fair to all streamlines, but does tend to produce shorter streamlines overall.. We believe it may be possible to achieve even better results by incorporating the similarity distance into a richer optimization framework with more legal moves for exploring the space of placements [13] . This is beyond the scope of this paper, but seems quite promising for the future.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A 48 Cycles/MB H.264/AVC Deblocking Filter Architecture for Ultra High Definition Applications In this paper, a highly parallel deblocking filter architecture for H.264/AVC is proposed to process one macroblock in 48 clock cycles and give real-time support to QFHD@60 fps sequences at less than 100 MHz. 4 edge filters organized in 2 groups for simultaneously processing vertical and horizontal edges are applied in this architecture to enhance its throughput. While parallelism increases, pipeline hazards arise owing to the latency of edge filters and data dependency of deblocking algorithm. To solve this problem, a zig-zag processing schedule is proposed to eliminate the pipeline bubbles. Data path of the architecture is then derived according to the processing schedule and optimized through data flow merging, so as to minimize the cost of logic and internal buffer. Meanwhile, the architecture's data input rate is designed to be identical to its throughput, while the transmission order of input data can also match the zig-zag processing schedule. Therefore no intercommunication buffer is required between the deblocking filter and its previous component for speed matching or data reordering. As a result, only one 24×64 two-port SRAM as internal buffer is required in this design. When synthesized with SMIC 130 nm process, the architecture costs a gate count of 30.2 k, which is competitive considering its high performance. The recent years have witnessed tremendous advances in digital video technology. While 1080HD has already become a standard of mainstream video applications such as TV broadcasting and home entertainment, even higher resolutions (e.g. QFHD and 8 K) and frame rates (e.g. 60 fps and 120 fps) are now targeted by high-end and next-generation services such like digital cinema and etc [1] . H.264/AVC [2] , as a powerful and popular video coding format, is sure to be a suitable tool for compressing the massive data stream of these new video sequences. However, as resolution and frame rate increase, design of real-time video codec LSIs is getting more and more challenging. Since it is usually inefficient as well as technically difficult to simply scale up the chip clock frequency due to timing, power and thermal issues, most efforts for improving the throughput of video codec LSIs have been concentrated on parallelism enhancement, or reducing the clock cycles required for processing a certain amount of video samples. Another contribution from the enhanced parallelism can be to power saving. If the required throughput is kept constant, supply voltage can be lowered as clock frequency scaled down. Finchelstein et al.'s H.264 video decoder [3] required only 14 MHz core frequency and 0.7-V supply voltage for processing a 720 p video sequence. Consequently its core power consumption is remarkably reduced to be less than 2 mW in 65 nm process. Deblocking filter is an important feature of H.264/AVC, which reduces blocking artifacts while contributing to bit rate saving. To evaluate the degree of blocking artifacts and the usefulness of H.264/AVC deblocking algorithm on the resolution of QFHD, the 3840×2160 "DucksTakingOff" sequence is encoded into various bit rates with deblocking on and off. For each bit stream, 10 frames are encoded with a picture structure of IBBPBBP... Then the equivalent bit rates at 60 fps are calculated by multiply the length of the bit stream by 6. Figure 1 illustrates the objective quality comparison, which shows the deblocking filter can contribute to improving the PSNR when the bit rate is lower than 400 Mbps. Another comparison in Fig. 2 shows the blocking artifacts are visible at 220 Mbps, and becoming very critical when bit rate decreases to 70 Mbps. After deblocking filter enabled, subjective quality can be improved, especially for the 70 Mbps case.. Deblocking filter is also one of the most data and computation intensive components in the H.264/AVC framework [4] , [5] . Furthermore, its algorithmic dependency between successive blocks adds to the difficulty of architecture design. Assuming that the whole LSI system is working at a typical clock frequency of 100 MHz, the theoretical time budget for processing the QFHD (3840×2160) 60 fps sequences should be 52 cycles for each macroblock, which is to be even more critical for higher specifications.. Many of the previous works on deblocking filter architectures have been targeted to standard HD applications, for which efficient utilization of one single edge filter should be fast enough [6] - [9] . However, it is difficult to apply these architectures to higher specifications, due to their throughput limitation of 192 cycles/MB. Some of the other designs have paid attention to parallelize 2 edge filters [10]- [13] . H.-Y. Lin et al.'s architecture [12] makes full use of its edge filters to process one MB in 96 clock cycles, at the cost of large on-chip SRAMs. Dang [11] proposed a low cost solution of application specific processor with no SRAMs, while its throughput is also lower, which is 192 cycles/MB that can be reached by some single filter solutions. Among these designs, Y.-C. Lin et al.'s architecture [10] achieves the highest average throughput by applying skip patterns to avoid some unnecessary filter operations. Its processing cycles for one MB vary from 48 to 100 according to the edge features. However, strictly speaking, the variable throughput approach is incapable of improving the worst-case performance. Meanwhile, to give play to this approach's advantage on average performance, large interconnection buffer is required between the deblocking filter and its neighboring components.. In this paper, 4 edge filters organized in 2 groups for simultaneously processing vertical and horizontal edges are applied in an H.264/AVC deblocking filter architecture to enhance its throughput. While parallelism increases, pipeline hazards arise owing to the latency of edge filters and data dependency of deblocking algorithm. To solve this problem, a zig-zag processing schedule is proposed for this architecture to eliminate the pipeline bubbles. Data path of the architecture is then derived according to the processing schedule and optimized through data flow merging, so as to minimize the cost of logic and internal buffer. Meanwhile, the architecture's data input rate is designed to be identical to its throughput, while the transmission order of input data can also match the zig-zag processing schedule. As a result, no intercommunication buffer is required between the deblocking filter and its previous component for speed matching or data reordering. Finally, the proposed architecture can process one macroblock with 48 cycles in a pipeline, so that real-time processing of QFHD@60 fps sequences can be supported with less than 100 MHz. In this paper, we apply 4 edge filters organized in 2 groups that simultaneously process vertical and horizontal edges, so as to enhance the throughput of a H.264/AVC deblocking filter architecture. While parallelism increases, pipeline hazard problems arise owing to the latency of edge filters and data dependency of deblocking algorithm. To solve this problem, a zig-zag processing schedule is adopted and pipeline bubbles are successfully eliminated. Data path of the architecture is then derived according to the processing schedule and optimized through data flow merging, so as to minimize the cost of logic and internal buffer. Meanwhile, the architecture's data input rate is designed to be identical to its throughput, while the transmission order of input data can also match the zig-zag processing schedule, so that no intercommunication buffer is required between the deblocking filter and its previous component for speed matching or data reordering. Finally, the proposed architecture can process one macroblock with 48 cycles in a pipeline, so that QFHD@60 fps sequences can be supported with a clock frequency of less than 100 MHz. As internal buffer, only a 24×64 two-port SRAM is needed in this design. When synthesized with SMIC 130 nm process, the architecture costs a gate count of 30.2 k, which is competitive considering its high performance. We also verified this design by simulating it in a previously designed HD video decoder.

O O O B B O O O O O O O O O O O O B B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O B B O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A REFINEMENT OF THE COMMON CAUSE PRINCIPLE Abstract. I study the interplay between stochastic dependence and causal relations within the setting of Bayesian networks and in terms of information theory. The application of a recently defined causal information flow measure provides a quantitative refinement of Reichenbach's common cause principle. The problem of system identification. Understanding the interplay and the function of a system's components generally requires not only phenomenological studies of global complex behaviour but also the study of the system's functional response to controlled experimental perturbations. Ideally, with a corresponding experimental design one aims at a complete identification of the system's mechanisms. In the context of biological systems this is clearly a problematic issue. On the one hand, a biological system may not be resistant to all experimental perturbations that would allow for its identification, and, on the other hand, the identity of the system may change as consequence of perturbations. Furthermore, in addition to these problems, there are also technology constraints dictated by the scientific and financial means at hand. In view of these limitations, one has to address the problem of specifying the kind of conclusions that can be drawn based on a particular set of feasible experiments. This requires a theoretical tool that is capable of modelling not only the system itself but also the data generating experimental perturbations.. Most system-theoretical models distinguish three levels of mechanistic specifications:. (1) specification of the system's units, (2) structural description of the relations between the units, (3) functional description of the units' interactions. Graph theory turns out to be very useful in providing a mathematical structure that serves as a model for levels (1) and (2) of the system description. In this model, the nodes of a graph represent the units of the system and the edges describe their relationship. In view of the model diversity currently present in the literature, an interpretation of edges strongly depends on the particular context. In this paper we follow Pearl's conceptual line [Pearl, 2000] by using edges as a qualitative representation of possible direct causal effects. Within Pearl's theory the functional description of level (3) is based on the so-called structural equation model originally developed in genetics [Wright, 1921] . In order to provide a self-consistent and coherent presentation I shall give a review of relevant concepts of Pearl's causality theory including the structural equation model (see the appendix).. In Pearl's causality theory, the concept of intervention, which is intended to capture controlled experimental perturbations, plays an essential role in modelling causal effects within the system. Although experimental intervention techniques such as knockout methods in genetics are of utmost importance for understanding function in biological systems, the functional interpretation of corresponding post-interventional observations is often based on heuristic arguments and needs further justification. Here, the interplay between causal effects and general stochastic associations is still a source of confusion. In order to identify associations and the way they are generated by cause-effect relations in a given system, observation and intervention are considered as elementary experimental operations. The identification of associations of system variables is a subject of classical statistics and requires only the observation of these variables, whereas the identification of cause-effect relations in general requires experimental intervention. Pearl's theory addresses the central problem of finding particular situations in which causal effects can be identified without any active experimental intervention based on purely non-interventional observations. It is quite surprising that, indeed, there are several criteria, including the back-door and front-door criteria, that provide such an identifiability test for causal effects [Pearl, 2000] . Unfortunately, the assumptions here require some structural information about the underlying network, which shows that replacement of interventional data by purely observational data is not possible without any cost.. If the structural information that is required in above-mentioned identifiability criteria is not available, some weaker statements on the cause-effect relations can still be made. For instance, Reichenbach's principle of common cause [Reichenbach, 1956] , which is phrased by the slogan no correlation without causation, identifies a class of possible causal relations between two variables A and B if they are stochastically dependent.. 1 It predicts that A is a cause of B, B is cause of A, or there is a common cause of A and B as shown in Figure 2 . Clearly, in this qualitative statement the particular value of stochastic dependence of A and B is not used. A quantitative version of this statement would provide more information about the system. This paper extends the common cause principle in two directions: On the one hand, the main results (Theorems 3 and 4, Corollary 1) refer to more than two variables, and, on the other hand, they make quantitative statements by relating the stochastic dependence of these variables to causal information flows introduced in [Ay & Polani, 2007] . The following instructive example illustrates the main idea of the paper.. An example. Consider, as shown in Figure 1 , two units A and B, where A has a direct causal effect on B but not vice versa.. A B Figure 1 In this situation, all stochastic dependence between A and B is due to the causal effect of A on B, and we can use the mutual information as measure for the causal information flow from A to B:. where H(B) denotes the entropy of B, H(B|A) denotes the conditional entropy of B given A, and I(A : B) is the mutual information of A and B (see Section 3). This definition is somewhat confusing. The suggestive notation of the left-hand side of (1) reflects the intuition that the causal information flow should be a directional quantity. On the other hand, mutual information is symmetric in A and B. If we just observe A and B and do not know the direction of the arrow there is no way to decide whether mutual information is an appropriate measure for the causal information flow from A to B. The situation becomes even more subtle if we have a common cause C of A and B which induces positive mutual information between A and B without having any causal information flow between A and B. Figure 2 illustrates three completely different causal structures that allow for observational equivalence on A and B:. Figure 2 Can we say anything about causal information flows in the network based on the observation of A and B only, without knowing the network structure? In order to illustrate in what sense this is indeed possible we define the total causal information flow for the three situations as sums of the (local) information flows along the arrows as defined in (1):. In all three cases the mutual information I(A : B) provides a lower bound for the total information flow:. The first two equalities in (2) trivially follow from the symmetry of mutual information, whereas the inequality is a consequence of the conditional independence structure of the network (c) which implies H(A, B|C) = H(A|C) + H(B|C), and therefore. Thus, in this example, without knowing the concrete underlying causal structure (a), (b), or (c) and corresponding mechanisms that generate the observed data distribution, we can give a lower bound for the total causal information flow in the network. This example represents a simplified quantitative version of Reichenbach's common cause principle which, in our setting infers positive causal information flow based on positive mutual information (stochastic dependence).. Organization of the paper. In the following Section 2 some concepts of Pearl's theory of causation are briefly outlined. This theory is based on a formal definition of intervention within the framework of Bayesian networks. Motivated by Reichenbach's principle of common cause, in Section 3 an optimal graphical criterion for the equivalence of intervention and observation will be provided. This criterion characterizes those cases in which stochastic dependence can be interpreted as causal information flow, a notion discussed in Section 4. The main Section 5 provides a lower bound for information flows in terms of the multi-information of random variables and shows how this extends the common cause principle. Finally, the appendix contains a motivation of Pearl's concept of intervention in terms of the structural equation model, and, furthermore, the proofs of the theorems of the paper.

O O O O B I I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O B O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A Survey on OFDM-Based Elastic Core Optical Networking frequency-division multiplexing (OFDM) is a modulation technology that has been widely adopted in many new and emerging broadband wireless and wireline communication systems. Due to its capability to transmit a high-speed data stream using multiple spectral-overlapped lower-speed subcarriers, OFDM technology offers superior advantages of high spectrum efficiency, robustness against inter-carrier and inter-symbol interference, adaptability to server channel conditions, etc. In recent years, there have been intensive studies on optical OFDM (O-OFDM) transmission technologies, and it is considered a promising technology for future ultra-high-speed optical transmission. Based on O-OFDM technology, a novel elastic optical network architecture with immense flexibility and scalability in spectrum allocation and data rate accommodation could be built to support diverse services and the rapid growth of Internet traffic in the future. In this paper, we present a comprehensive survey on OFDM-based elastic optical network technologies, including basic principles of OFDM, O-OFDM technologies, the architectures of OFDMbased elastic core optical networks, and related key enabling technologies. The main advantages and issues of OFDM-based elastic core optical networks that are under research are also discussed.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Escape planning in realistic fire scenarios with Ant Colony Optimisation Abstract An emergency requiring evacuation is a chaotic event, filled with uncertainties both for the people affected and rescuers. The evacuees are often left to themselves for navigation to the escape area. The chaotic situation increases when predefined escape routes are blocked by a hazard, and there is a need to re-think which escape route is safest. This paper addresses automatically finding the safest escape routes in emergency situations in large buildings or ships with imperfect knowledge of the hazards. The proposed solution, based on Ant Colony Optimisation, suggests a near optimal escape plan for every affected personconsidering dynamic spread of fires, movability impairments caused by the hazards and faulty unreliable data. Special focus in this paper is on empirical tests for the proposed algorithms. This paper brings together the Ant Colony approach with a realistic fire dynamics simulator, and shows that the proposed solution is not only able to outperform comparable alternatives in static and dynamic environments, but also in environments with realistic spreading of fire and smoke causing fatalities. The aim of the solutions is usage by both individuals, such as from a personal smartphone of one of the evacuees, or for emergency personnel trying to assist large groups from remote locations. Evacuation planning is challenging due to the typically chaotic and un-organised situation occurring in a crisis situation. Unfortunately, decision makers often have an incomplete picture of hazards and potential escape routes. The situation is further complicated by the fact that people affected are often left alone without any contact with rescue personnel, or let alone a reliable overview of the ongoing crisis situation. In addition, the chaotic and dynamic nature of crisis situations quickly changes which path is the best escape route as hazards, such as fires, rapidly develop.. There is no doubt that decision making in crisis situations needs to be timely to minimize the consequences of hazards at hand. However, it is often difficult for the people affected to determine what are the best decisions in an evacuation situation. In fact, in most situations the evacuees are not aware of which path to follow for an escape. This is because evacuees either received insufficient information from the rescuers or are unfamiliar with the layout of the affected compound and locations of the hazards. Similarly, the emergency personnel often lack an overview of where people are located and which rooms are affected by hazards. This makes evacuation planning particularly difficult [1, 2] .. This paper is part of a larger project 1 working on using smartphone technologies in emergency situations. In this project, the smartphones are used to communicate to and from people affected by crisis situations. This enables rescue personnel to get an overview of the people escaping such as information on how they move. Further, the rescuers can communication how to best escape to the evacuees. In addition, the sensor information available in smartphones will communicate presence of hazards using camera, GPS, gyroscope, etc. This provides a threat map which is support both the rescue personnel and as a method to automatically determine the best escape routes [3] .. The project has three main steps outlined in the sections below:. 1. Collect information. 2. Calculate escape plan. 3. Communicate the plan to the people affected. This paper presents an application using Ant Colony Optimisation (ACO) for finding safe escape routes in emergency situations. ACO is used in five distinct environments. Firstly, ACO operates in a stationary environment where it quickly reaches a near optimal solution. Secondly, ACO is run in dynamic situations where hazards rapidly change. Further, ACO is evaluated in three realistic environments: fire on board a ship without smoke, fire on board a ship with smoke and fire on board a ship in an ad-hoc network scenario. The realistic scenarios are achieved through set ups from the Fire Dynamics Simulator tool. In all tested scenarios ACO is empirically able to reach a near optimal solution. In some setups, ACO with 1,000 ants find the optimal route in almost all situations. This leads to the unproven assumption that given enough ants, ACO will always find the optimal escape route. In our further work, we plan to formally verify that whether assumption is correct.. In real evacuation situations, most people escape together in groups with their family and friends. Further, panic tends to spread in emergency situations which in practice means that many will not follow any provided escape routes. These situations are also planned for further testing.. Additionally, we plan to examine methods for faster convergence of ACO including testing the min-max variants of ACO, and examining the possibilities of a multi layered approach.. Most importantly, we are currently implementing a smartphone application for safe escape planning that will use the ACO planning. Jaziar Radianti received her Ph.D in System Dynamics from University of Bergen, Norway. Dr. Radianti is a researcher for CIEM (Centre for Integrated Emergency Management) at Department of ICT, University of Agder, Norway. She has served as a reviewer for numerous international conferences. Her research interests include the application of simulation approaches, especially system dynamics, fire dynamics, and Bayesian network modeling for disaster and crisis management. Currently, she is also developing an interest toward research in mobile sensing, and its intersections with human centric computing, machine learning techniques and the application for decision support in a crisis. The simulations were carried out with fire starting in the middle of the first floor, and kept for 600 simulated seconds. 7 A visual representation of the temperature is shown in Fig. 6 . Note that the fire reaches one corridor at ∼150 s, and both corridors at close to ∼500 s.. The setup is slightly different from previous experiments. In this case we have a fire that spreads out over a time frame of 600 s. These seconds are simulated, and for each second a given number of consecutive ants are released. This means that every second is in itself a distinct scenario similar to the complete experiments in Figs. 2 and 3 . This facilitates convergence of the algorithm so that the route s * represents the "best" possible result. 8 Further, in the follow empirical results three variants of ACO are used: with 10, 100 and 1,000 ants. This is done to show the convergence of the algorithms -how many ants are needed to find near optimal solutions. Thus, in contrast to Figs. 2 and 3 the following figures show snapshots of the algorithm after 10,100 and 1,000 ants for 600 different scenarios.. Keep in mind that all simulations run 1,000 times, and the objective is still the same, finding a route s to minimize f (s). I.e. find the safest possible escape route. Figure 7 shows the behaviour where the evacuees starting position is manipulated so that they all start from a fixed location on board the ship, the room with id 229. The room chosen to be particularly difficult to escape from is on the opposite side of the ship's escape area on the second floor - 7 Two other simulations were carried out with similar setup but with fires starting in the corner of the first floor and the middle of the second floor. The results were similar to those shown in this paper -and are therefore left out. 8 The ants do not have any knowledge of the spread of the fire other than what they observe through the function h(v i , t), so "best" means most viable seen from the small windows available from the ants. In practice, data from fixed and mobile fire sensors will populate h(v i , t).. so that the fire always occurs between the evacuees and the escape area. Hence, the first room in the route s, the source v s , is always "room 229". Note that there still exists an escape route, it is just more difficult to find.. Conversely, Fig. 8 shows the behaviour where the evacuees start from random locations on board the ship. Hence, the first room in the route s is uniformly randomly selected among all v i ∈ V . It is noteworthy the difficult situation in Fig. 7 is remarkably similar to the over all situation in Fig. 8 , only with an over all higher value for f (s) for all algorithms. A conclusion to be drawn from this is that even in the difficult situations ACO is able to find safe escape routes, but requires more ants to reach convergence.

B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O