Organizational

structure

and

the

periphery

of

the

gene

regulatory

network

in

B-cell

lymphoma

Background:

The

physical

periphery

of

a

biological

cell

is

mainly

described

by

signaling

pathways

which

are

triggered

by

transmembrane

proteins

and

receptors

that

are

sentinels

to

control

the

whole

gene

regulatory

network

of

a

cell.

However,

our

current

knowledge

about

the

gene

regulatory

mechanisms

that

are

governed

by

extracellular

signals

is

severely

limited.

Results:

The

purpose

of

this

paper

is

three

fold.

First,

we

infer

a

gene

regulatory

network

from

a

large-scale

B-cell

lymphoma

expression

data

set

using

the

algorithm.

Second,

we

provide

a

functional

and

structural

analysis

of

the

largest

connected

component

of

this

network,

revealing

that

this

network

component

corresponds

to

the

peripheral

region

of

a

cell.

Third,

we

analyze

the

hierarchical

organization

of

network

components

of

the

whole

inferred

B-cell

gene

regulatory

network

by

introducing

a

new

approach

which

exploits

the

variability

within

the

data

as

well

as

the

inferential

characteristics

of

As

a

result,

we

find

a

functional

bisection

of

the

network

corresponding

to

different

cellular

components.

Conclusions:

Overall,

our

study

allows

to

highlight

the

peripheral

gene

regulatory

network

of

B-cells

and

shows

that

it

is

centered

around

hub

transmembrane

proteins

located

at

the

physical

periphery

of

the

cell.

In

addition,

we

identify

a

variety

of

novel

pathological

transmembrane

proteins

such

as

ion

channel

complexes

and

signaling

receptors

in

B-cell

lymphoma.

With

the

increasing

quality

of

network

inference

algorithms,

we

are

heading

toward

the

next

major

challenge

we

are

facing

in

the

post-genomic

era,

namely:

What

do

the

inferred

networks

mean?

An

analysis

of

the

hierarchical

organization

of

a

network

is

just

one

aspect

thereof,

but

we

think,

an

import

one.

Due

to

the

fact

that

one

can

study

the

hierarchy

among

genes,

pathways,

subnetworks

or

combinations

thereof

the

complexity

of

this

problem

might

be

unprecedented.

The

bootstrap

approach

presented

in

this

paper

represents

a

simple,

yet,

flexible

method

in

order

to

tame

the

complexity

of

the

problem

resulting,

additionally,

in

an

interpretable

structure.

In

this

article,

we

inferred

a

B-cell

gene

regulatory

network

from

B-cell

lymphoma

gene

expression

data

using

the

algorithm

We

found

that

the

inferred

B-cell

gene

regulatory

network

is

characterized

by

individual

network

components

that

are

organized

by

smaller

interconnected

network

modules

with

intramodular

hub

genes.

Further,

we

found

that

the

giant

connected

component

of

the

network

is

composed

of

genes

which

show

a

significant

enrichment

for

plasma

membrane

proteins

that

are

involved

in

G

protein

signaling

pathways

and

ion

channel

complexes.

From

the

literature,

it

is

known

that

ion

channels

play

a

key

role

for

the

signal

transduction

mechanism

in

lymphocytes

Additionally,

we

found

that

the

largest

components

of

the

entire

network

can

be

categorized

into

two

major

classes.

The

first

class,

including

the

largest

network

component,

is

enriched

by

genes

that

are

located

at

the

membrane

and

the

extracellular

space

at

the

physical

periphery

of

the

cell

whereas

the

second

class

comprises

network

components

located

in

the

intracellular

organelles

such

as

in

the

cytoplasm,

nucleus

and

mitochondrion.

Further,

the

hub

genes

of

the

B-cell

gene

regulatory

network

were

identified

to

play

crucial

roles

in

cell

signaling,

adhesion

and

cell

proliferation

processes..

It

is

believed

that

B-cell

lymphoma

subtypes

show

characteristic

gene

expression

profiles

of

B-cells

that

are

arrested

in

specific

developmental

stages

The

emergence

of

a

lymphoma

phenotype

is

thus

understood

to

result

from

an

impairment

of

pathways

that

control

B-cell

differentiation,

proliferation

and

apoptosis

processes

The

organizational

structure

of

gene

regulatory

networks

is

a

rich

source

of

information

to

study

specific

molecular

mechanisms

of

B-cell

lymphoma.

However,

the

combination

of

observational

and

experimental

conditions

from

a

variety

of

different

B-cell

lymphoma,

including

transformed

and

untransformed

cells,

as

for

our

data

does

not

allow

to

infer

a

gene

regulatory

network

for

one

particular

subtype

of

B-cell

lymphoma.

Thus

we

are

of

the

opinion

that

our

inferred

B-cell

gene

regulatory

network

represents

an

average

representation

of

B-cell

lymphoma

reflecting

different

phenotypic

subtypes

with

which

the

information

conveyed

by

the

gene

expression

values

is

associated..

In

it

has

been

demonstrated

that

not

all

regions

within

a

network

can

be

inferred

with

the

same

inference

accuracy.

That

means,

the

inference

of

networks

is

heterogeneous

with

respect

to

distinct

edges

in

the

network.

It

has

been

shown

that

moderately

interconnected

genes

are

easier

to

infer.

This

corresponds

to

the

edges

of

linearly

connected

genes

and

the

edges

toward

the

leaf

nodes

of

the

network

that

are

at

the

'periphery'

of

the

network.

The

results

in

have

been

obtained

for

simulated

data.

However,

for

a

real

biological

gene

regulatory

network

it

was

unclear

what

genes

correspond

to

the

periphery

of

this

cellular

network.

In

contrast,

in

this

paper

we

demonstrated

that

the

periphery

of

the

inferred

B-cell

gene

regulatory

network

is

centered

around

transmembrane

proteins

and

the

linear

parts

of

the

gene

regulatory

network

correspond

to

signaling

pathways

and

transmembrane

receptor

or

ion

channel

proteins

involved

in

signaling

cascades.

We

would

like

to

note

that

these

transmembrane

proteins

could

form

putative

drug

targets

for

B-cell

lymphoma..

The

algorithm

selects

at

most

one

edge

for

each

gene,

having

maximum

mutual

information

value.

Therefore,

this

algorithm

intends

to

capture

the

conservative

causal

core

of

the

whole

regulatory

network

only.

This

is

in

contrast

to

many

other

network

inference

methods

For

this

reason,

it

is

no

surprise

that

a

previous

analysis

of

the

same

data

set

employing

a

different

network

inference

method

found

that

their

inferred

regulatory

network

is

governed

by

major

hub

genes,

which

mark

key

regulators

such

as

transcription

factors

In

particular,

the

network

inferred

by

ARACNE

consisted

of

edges

and

their

major

hub

genes

are

reported

to

be

cell

cycle

regulators.

In

contrast

to

these

results,

we

found

by

our

analysis

a

network

with

genes

and

edges

enriched

for

signaling

pathways

In

the

following

section

we

present

the

methods

and

the

data

used

for

our

analysis.

O O O O O O O B I I O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O B I B I I O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I I O O O O O B I O O O O O O O O O O O B I I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


ROBUST PROFILE VERIFICATION SCHEME BASED ON EAR REFERENCE COORDINATE SYSTEM In this paper, a novel profile verification scheme based on the ear, contour lines and feature points from head profile images is proposed. From the side image of a human head taken by a CCD camera, the contour line of the image is extracted by the simple filling method and morphological filtering. Based on the reference coordinate set by using ear images enrolled in the database a priori, the features of the contour line such as the tip of the nose, bottom of the nose, eye point and so on are extracted. The contour of the given image is divided into segments by utilizing the extracted feature points. The directional code of each segment is proposed and the length of the segment is computed for generating the feature vector. The verification of the given profile is performed by matching these feature vectors with those of the enrolled database. The experimental results show validity of the developed method for feature extraction and profile verification. In recent years, the personal identification and recognition techniques based on biometric features become a very important tool to share and secure personal information. Biometric features which can be used for identifying a person must show the following characteristics: the uniqueness of the extracted features for each person and the invariance of the features about time. For a human body, many kinds of biometric features with those characteristics exist.. Fingerprints have been studied as one of the important biometric features for the identification of individuals since ancient times. 9 In the fingerprint verification method, patterns of fingerprints such as the bifurcation, ending points, upper core, valley and ridge are utilized for the unique features of an individual. Among biometric features of a human body, the geometry of a human hand or the pattern of the hand vein which is the blood tube on the back of a human fist, has been developed as a feature for verification of a person. 4, 13 In the verification system with hand geometry, the width and length of the fingers, area of the palm, and the length between two joints of the finger are listed as geometry features.. Each individual has a unique and robust iris pattern, thus it has been considered as good information for the identification of individuals. Noticeable studies have been done by Daugman 5 and Wildes 16 in recent years. Also, numerous algorithms have been proposed for automatic personal identification using facial images, especially the side view of a human head. 3, 7, [10] [11] [12] Harmon et al. have investigated personal identification using profile images and 86 ∼ 100% accuracy of the correct verification has been confirmed. 7 Okamoto et al. have proposed a profile identification method by using a color difference image with the specific background. 11 In this method, several side views of a human head are captured to cope with difficulties in changes of the extracted features. Chen, Qi et al. have researched and suggested their methods for profile verification in various forms. In this paper, a novel algorithm of feature extraction and verification based on the profile image is presented. The proposed algorithm has two main contributions for profile verification. The first is to set a new coordinate system by employing the ear image and to propose the directional code instead of the angle value of the contour. The second is the two-stage verification scheme. By utilizing this scheme, the consumed time for verification can be reduced significantly when the enrolled individuals are great in number. Because the proposed algorithm sorts out probable candidates among all enrolled persons by utilizing the ear database images and comparing these with the given input image in the first step of the two-stage verification scheme.. The experimental results show the usefulness of the developed algorithm. From the results, it can be seen that the extracted features are robust against the change of the feature points caused by the direction change of the camera axis or the obscure lighting conditions. The personal identification, based on the matching function which is taken into account by the error of the ear matching as the weighting factors, indicates a very high rate of correct identification. The proposed techniques are reliable and prospective for an individual verification system. Also, the proper lighting conditions and the configurations for capturing the side view of a human head or the profile image are now under study.. For further work, we should consider the consumed time for the overall verification and investigate a faster block or feature matching of the ear image to reduce the verification time effectively.

O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O B I O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Reduced Basis Decomposition: a Certified and Fast Lossy Data Compression Algorithm Dimension reduction is often needed in the area of data mining. The goal of these methods is to map the given high-dimensional data into a lowdimensional space preserving certain properties of the initial data. There are two kinds of techniques for this purpose. The first, projective methods, builds an explicit linear projection from the high-dimensional space to the low-dimensional one. On the other hand, the nonlinear methods utilizes nonlinear and implicit mapping between the two spaces. In both cases, the methods considered in literature have usually relied on computationally very intensive matrix factorizations, frequently the Singular Value Decomposition (SVD). The computational burden of SVD quickly renders these dimension reduction methods infeasible thanks to the ever-increasing sizes of the practical datasets.In this paper, we present a new decomposition strategy, Reduced Basis Decomposition (RBD), which is inspired by the Reduced Basis Method (RBM). Given X the high-dimensional data, the method approximates it by Y T (≈ X) with Y being the low-dimensional surrogate and T the transformation matrix. Y is obtained through a greedy algorithm thus extremely efficient. In fact, it is significantly faster than SVD with comparable accuracy. T can be computed on the fly. Moreover, unlike many compression algorithms, it easily finds the mapping for an arbitrary "out-of-sample" vector and it comes with an "error indicator" certifying the accuracy of the Dimension reduction is ubiquitous in many areas ranging from pattern recognition, clustering, classification, to fast numerical simulation of complicated physical phenomena. The fundamental question to address is how to approximate a n-dimensional space by a d-dimensional one with d n. Specifically, we are given a set of high-dimensional data. and the goal is to find its low-dimensional approximation. with reasonable accuracy. There are two types of dimension reduction methods. The first category consists of "projective" ones. These are the linear methods that are global in nature, and that explicitly transform the data matrix X into a low-dimensional one by Y = T X. The leading examples are the Principal Component Analysis (PCA) and its variants. The methods in the second category act locally and are inherently nonlinear. For each sample in the high-dimensional space (e.g. each column of X), they directly find their lowdimensional approximations by preserving certain locality or affinity between nearby points.. In this paper, inspired by the reduced basis method (RBM), we propose a linear method called "Reduced Basis Decomposition (RBD)". It is much faster than PCA/SVD-based techniques. Moreover, its low-dimensional vectors are equipped with error estimator indicating how close they are approximating the high-dimensional data. RBM is a relative recent approach to speed up the numerical simulation of parametric Partial Differential Equations (PDEs) [16, 19, 20, 6, 5] . It utilizes an Offline-Online computational decomposition strategy to produce surrogate solution (of dimension N ) in a time that is of orders of magnitude shorter than what is needed by the underlying numerical solver of dimension N N (called truth solver hereafter). The RBM relies on a projection onto a low dimensional space spanned by truth approximations at an optimally sampled set of parameter values [1, 8, 17, 18, 13] . This low-dimensional manifold is generated by a greedy algorithm making use of a rigorous a posteriori error bounds for the field variable and associated functional outputs of interest which also guarantees the fidelity of the surrogate solution in approximating the truth approximation.. The RBD method acts in a similar fashion. Given the data matrix X as in (1), it iteratively builds up Y (2) whose column space approximates that of X. It starts with a randomly selected column of X (or a user input if existent). At each step where we have k vectors {y 1 , . . . , y k }, the next vector y k+1 is found by scanning the columns of X and locating the one whose error of projection into the current space span{y 1 , . . . , y k } is the largest. This process is continued until the maximum projection/compression error is small enough or until the limit on the size of the reduced space is reached. An important feature is an offline-online decomposition that allows the computation of the compression error, and thus the cost of locating y k+1 , to be independent of (the potentially large) m.. This paper is organized as follows. In Section 2, we review the background material, mainly the RBM. Section 3 describes the reduced basis decomposition algorithm and discuss its properties. Numerical validations are presented in Section 4, and finally some concluding remarks are offered in Section 5. This paper presents and tests an extremely efficient dimension reduction algorithm for data processing. It is multiple times faster than the SVD/PCAbased algorithms. What makes this possible is a greedy algorithm that iteratively builds up the reduced space of basis vectors. Each time, the next dimension is located by exploring the errors of compression into the current space for all data entries. Thanks to an offline-online decomposition mechanism, this searching is independent of the size of each entry. Numerical results including one concerning a real world face recognition problem confirm these findings. The reduced basis method was developed for use with finite element methods to numerically solve PDEs. We assume, for simplicity, that the problems (usually parametric partial differential equations (PDE)) to simulate are written in the weak form: find u(µ) in an Hilbert space X such that a(u(µ), v; µ) = f (v; µ), ∀v ∈ X where µ is an input parameter. These simulations need to be performed for many values of µ chosen in a given parameter set D. In this problem a and f are bilinear and linear forms, respectively, associated to the PDE (with a N and f N denoting their numerical counterparts). We assume that there is a numerical method to solve this problem and the solution u N , called the "truth approximation" or "snapshot", is accurate enough for all µ ∈ D.. The fundamental observation utilized by RBM is that the parameter dependent solution u N (µ) is not simply an arbitrary member of the infinitedimensional space associated with the PDE. Instead, the solution manifold M = {u N (µ), µ ∈ D} can typically be well approximated by a lowdimensional vector space. The idea is then to propose an approximation of. pre-computed truth approximations corresponding to the parameters {µ 1 , . . . , µ N } judiciously selected according to a sampling strategy [13] . For a given µ, we now solve in W N for the reduced solution u (N ) (µ). The online computation is N -independent, thanks to the assumption that the (bi)linear forms are affine 2 and the fact that they can be approximated by affine (bi)linear forms when they are nonaffine [2, 10] . Hence, the online part is very efficient. In order to be able to "optimally" find the N parameters and to assure the fidelity of the reduced basis solution u (N ) (µ) to approximate the truth solution u N (µ), we need an a posteriori error estimator ∆ N (µ) which involves the residual r(v,. , v; µ) and stability information of the bilinear form [12, 14, 19, 20, 21] . With this estimator, we can describe briefly the classical greedy algorithm used to find the N parameters µ 1 , . . . , µ N and the space W N . We first randomly select one parameter value and compute the associated truth approximation. Next, we scan the entire (discrete) parameter space and for each parameter in this space compute its RB approximation u (N =1) and the error estimator ∆ 1 (µ). The next parameter value we select, µ 2 , is the one corresponding to the largest error estimator. We then compute the truth approximation and thus have a new basis set consisting of two elements. This process is repeated until the maximum of the error estimators is sufficiently small.. The reduced basis method typically has exponential convergence with respect to the number of pre-computed solutions [15, 4, 3] . This means that the number of pre-computed solutions can be small, thus the computational cost reduced significantly, for the reduced basis solution to approximate the finite element solution reasonably well. The author and his collaborators showed [7] that it works well even for a complicated geometric electromagnetic scattering problem that efficiently reveals a very sensitive angle dependence (the object being stealthy with a particular configuration).

O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Steepest descent with momentum for quadratic functions is a version of the conjugate gradient method It is pointed out that the so called momentum method, much used in the neural network literature as an acceleration of the backpropagation method, is a stationary version of the conjugate gradient method. Connections with the continuous optimization method known as heavy ball with friction are also made. In both cases, adaptive (dynamic) choices of the so called learning rate and momentum parameters are obtained using a control Liapunov function analysis of the system. q The backpropagation with momentum algorithm (BPM) has been much analyzed in the neural network literature and even compared with other methods, in particular, the conjugate gradient (CG) method (Kamarthi & Pittner, 1999; Yu & Chen, 1997) . However, although it has been noticed in these two papers that BPM is actually a stationary (or time-invariant) version of the well known CG algorithm, this fact does not seem to have been enunciated explicitly enough in order to have permeated the neural network community, since papers continue to be written on the analysis of BPM without mentioning or exploiting this connection (Qian, 1999; Torii & Hagan, 2002) . The purpose of this note is to point out this fact as well as generalize the results of both these papers to the time-varying or dynamic case. In particular, for a quadratic error function, the choice of learning and momentum parameters that has been referred to as 'optimally tuned' is shown to be exactly equivalent to using the CG algorithm.. For brevity, this note will focus on the contributions of Qian (1999) and Torii and Hagan (2002) which are recent and clearly written time-invariant analyses of the BPM method, which has been extensively analyzed, both theoretically and experimentally (see, for example, Hagiwara and Sato (1995) , Kamarthi and Pittner (1999) , Phansalkar and Sastry (1994) , Yu and Chen (1997) , and Yu, Chen, and Cheng (1995) and references therein).. In Torii and Hagan (2002) , a detailed analysis of the convergence of the BPM method is carried out for the case where a quadratic error function is to be analyzed. In Qian (1999) , the BPM algorithm is shown to result from the discretization of a continuous-time ordinary differential equation that models the movement of Newtonian particles through a viscous medium in a conservative force field. These authors do not make connections between their results and the CG algorithm or earlier results on continuous optimization.. This paper points out that the BPM method presented in Torii and Hagan (2002) for fixed momentum and steepest descent parameters (or gains) is actually a special case of the more general CG method, in which both these parameters are chosen dynamically in feedback form. It can be shown, in fact, that a control Liapunov function approach leads, in a straightforward manner, to the CG choice of these parameters (Bhaya & Kaszkurewicz, 2002) . In respect of Qian's analogy between Newtonian particles and the BPM method, it is pointed out that there are results on so called continuous optimization, starting as far back as Polyak (1964) , Tsypkin (1971) , and, more recently, in Attouch, Goudou, and Redont (2000) and Polyak (1987) , where this method is referred to as the 'heavy ball with friction' (HBF) method. It turns out that, in fact, both Qian's continuous version of BPM and a continuous version of the CG method can be regarded as the HBF method. However, we point out that it is more fruitful to introduce a continuous version of the CG method that is strictly analogous to the discrete CG algorithm and to regard the learning rate and momentum factor parameters as inputs to this system. When this is done, the resulting continuous-time system is a pair of coupled bilinear systems that permits a simple control Liapunov function analysis, furnishing state-dependent (i.e. feedback) choices of the learning rate and momentum factor that provide global asymptotic stability.. Finally, the control Liapunov function analysis technique used in this paper is very general and can be used to analyze other proposals for tuning parameters in BPM algorithms. This paper establishes various connections between the CG algorithm and the backpropagation algorithm with momentum acceleration. In particular, a general control Liapunov function approach to the analysis as well as design of BPM type algorithms is proposed. A continuous version of the CG algorithm is put forward and it is shown how to choose the parameters (that correspond to the learning rate and momentum factor) in state feedback form in order to guarantee global asymptotic stability of this system, implying convergence of the error to zero. This result is new and should prove to be of value in understanding the theoretical properties of backpropagation type algorithms; specifically in understanding the role of the learning rate and momentum factor in modifying convergence properties. In conclusion, we quote a paragraph from Alber (1971) that is as relevant today (with some minor changes in the buzzwords) as when it was written 30 years ago: "The increasing interest in continuous-descent methods is due firstly to the fact that tools for the numerical solution of systems of ordinary differential equations are now well developed and can thus be used in conjunction with computers; secondly, continuous methods can be used on analog computers ( ¼ neural networks); thirdly, theorems concerning the convergence of these methods and theorems concerning the existence of solutions of equations and of minimum points of functionals are formulated under weaker assumptions than is the case for the analogous discrete processes" (Parentheses ours). Similar justification for the consideration of continuous versions of well known discrete-time algorithms can be found in Chu (1988) and Chu (1992) .

B I O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I I O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O B O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


BioPhytMol:

a

drug

discovery

community

resource

on

anti-mycobacterial

phytomolecules

and

plant

extracts

Background:

Tuberculosis

(TB)

is

the

second

leading

cause

of

death

from

a

single

infectious

organism,

demanding

attention

towards

discovery

of

novel

anti-tubercular

compounds.

Natural

products

or

their

derivatives

have

provided

more

than

of

all

existing

drugs,

offering

a

chemically

diverse

space

for

discovery

of

novel

drugs.

Description:

BioPhytMol

has

been

designed

to

systematically

curate

and

analyze

the

anti-mycobacterial

natural

product

chemical

space.

BioPhytMol

is

developed

as

a

drug-discovery

community

resource

with

anti-mycobacterial

phytomolecules

and

plant

extracts.

Currently,

it

holds

entries

including

plant

families

genera

and

species)

from

global

flora,

manually

curated

from

literature.

In

total,

there

are

phytomolecules

(with

structures)

curated

against

target

mycobacteria.

Multiple

analysis

approaches

have

been

used

to

prioritize

the

library

for

drug-like

compounds,

for

both

whole

cell

screening

and

target-based

approaches.

In

order

to

represent

the

multidimensional

data

on

chemical

diversity,

physiochemical

properties

and

biological

activity

data

of

the

compound

library,

novel

approaches

such

as

the

use

of

circular

graphs

have

been

employed.

Conclusion:

BioPhytMol

has

been

designed

to

systematically

represent

and

search

for

anti-mycobacterial

phytochemical

information.

Extensive

compound

analyses

can

also

be

performed

through

web-application

for

prioritizing

drug-like

compounds.

The

resource

is

freely

available

online

at

BioPhytMol

is

a

drug-discovery

community

resource

with

a

database

of

anti-mycobacterial

phytomolecules

and

plant

extracts.

This

a

unique

resource

that

encapsulates

important

information

such

as

their

bioactivity

values

(MIC),

target

bacteria,

percentage

inhibition

of

mycobacterial

growth

and

most

importantly

two

dimensional

as

well

as

three

dimensional

structures

for

purified

anti-mycobacterial

phytomolecules.

Structure

based

searching

and

comparison

against

various

drug

classes

aids

to

the

features

of

platform

to

speed

up

the

anti-mycobacterial

drug

discovery.

BioPhytMol

is

a

drug-discovery

community

resource

with

a

database

of

anti-mycobacterial

phytomolecules

and

plant

extracts

with

analysis

tools

integrated

into

a

web-application.

The

database

is

based

on

extensive

literature

search

and

curation

and

is

the

only

searchable

resource

on

anti-mycobacterial

phytomolecules.

In

addition

to

the

data

on

plant

extracts

or

the

purified

active

ingredients,

the

structure,

bioactivity

and

experimental

details

are

also

captured

systematically

through

a

well-defined

data

structure.

This

platform

is

designed

to

perform

analysis

based

both

on

chemical

structure

and

biological

properties

which

is

the

primer

for

any

early

stage

drug

discovery

process.

To

facilitate

structure-based

analysis,

BioPhytMol

has

been

equipped

with

structure-based

search

options,

which

may

be

used

to

compare

any

new

compound

with

the

BioPhytMol

database,

as

also

with

anti-TB

drugs

and

compounds

reported

in

DrugBank

in

various

categories.

These

searches

are

expected

to

reveal

shared/distinct

structural

features

of

the

new

and

known

anti-TB

drugs

with

that

of

other

drug-like

libraries

including

small

molecules

and

nutraceuticals.

It

is

interesting

to

observe

that

there

are

cases

where

a

compound

has

been

reported

to

show

activity

against

multiple

species

of

mycobacteria.

This

feature

of

database

may

help

in

identification

of

the

structural

features

responsible

for

highly

selective

or

broad-spectrum

activity

of

phytomolecules

against

different

mycobacterial

strains.

Similarly,

the

data

and

the

analysis

tools

also

enable

identification

of

structural

analogs

of

reported

anti-TB

compounds

and

inhibitors

of

specific

drug

targets

in

Mtb..

A

revised

ontology

and

chemical

classification

is

also

proposed

as

part

of

systematic

analysis

of

the

compound

library.

This

is

important

for

data-interoperability

and

shared

knowledgebase.

The

revised

chemical

classification,

though

not

extensive,

allows

the

chemists

to

search

for

compounds

with

specific

chemical

features

like

aromaticity

or

functional

groups

like

ketone

or

classes

like

specific

sugar

moieties,

thus,

making

the

platform

useful

for

researchers

from

multiple

disciplines..

A

highlight

of

the

study

is

the

representation

of

multidimensional

data

through

circular

graphs

that

allow

interpretation

of

various

chemical

properties

with

respect

to

each

other

and

with

bioactivity

data.

CIRCOS

has

been

used

to

generate

these

graphs.

CIRCOS

is

a

very

popular

tool

among

genomics

and

genetics

community

to

represent

the

physical

map

and

annotation

of

circular

genomes.

For

this

study,

CIRCOS

has

been

customized

to

represent

the

chemical

and

biological

properties

of

the

compound

library.

The

same

approach

can

be

used

to

represent

any

set

of

properties

and

is

expected

to

aid

in

interpretation

of

multidimensional

data..

The

real

success

of

a

database

lies

in

its

regular

updation.

Database

updation

and

curation

is

an

on-going

activity.

To

ensure

that

BioPhytMol

is

regularly

updated,

a

structured-wiki

platform

has

been

designed.

This

platform

allows

community

curation

of

new

datasets

as

well

as

modification

of

the

existing

one

for

any

ambiguities.

The

submitters

are

duly

acknowledged

for

their

contribution

as

each

annotation

or

entry

is

author

and

time

stamped.

The

curators

should

register

through

the

OSDD

portal

and

obtain

Sysborg

OpenID.

This

ID

can

then

be

used

to

update

the

database

through

online

form.

However,

database

changes

are

committed

only

when

the

submission

is

quality

checked

for

correctness

and

format.

Efforts

are

ongoing

to

extend

the

search

base

of

BioPhytMol

as

well

as

to

include

batch

query

options

where

multiple

structures

can

be

searched

using

the

similarity

search

tool

for

their

corresponding

similar

structures.

According

to

the

WHO

Global

Tuberculosis

report,

TB

alone

was

responsible

for

nearly

million

deaths

and

million

new

cases

in

The

current

TB

treatment

regimen

is

lengthy

and

multi-therapy

based.

Over

time,

TB

causing

bacteria

i.e.,

Mycobacterium

tuberculosis

(Mtb)

has

acquired

resistance

against

existing

antibiotics

leading

to

multi-drug

resistant

(MDR),

extensively

drug

resistant

(XDR)

and

totally

drug-resistant

(TDR)

forms

The

long-duration

therapies

also

possess

various

sideeffects

and

toxicity

issues

Bedaquiline

has

been

recently

approved

by

United

States

Food

and

Drug

Administration

(FDA)

in

December

almost

after

years

of

the

discovery

of

the

last

anti-tubercular

(anti-TB)

drug.

This

drug

comes

with

a

black

box

warning

of

induction

of

arrhythmia

followed

by

the

risk

of

developing

resistance

if

used

improperly

There

are

other

anti-TB

drugs

too

in

various

stages

of

development

However,

there

is

a

need

to

discover

novel,

highly

effective,

less-toxic

anti-TB

drugs

having

new

mechanism

of

action.

There

is

an

increasing

demand

to

explore

new

chemical

space

given

the

emergence

of

drug-resistance

even

for

newly

developed

therapies.

Exploring

natural

product

(NP)

space

may

prove

to

be

a

valuable

source

of

new

and

less-toxic

anti-TB

drugs

Analysis

of

the

NP

chemical

space

has

led

to

the

identification

of

novel

scaffolds

and

increased

chance

of

discovering

compounds

possessing

novel

mechanism

of

action

The

overall

chemical

space

covered

by

synthetic

compounds

is

limited

as

compared

to

natural

compounds

given

their

higher

chemical

diversity

More

than

of

FDA

approved

drugs

have

been

either

NPs

or

NP

derivatives

For

example,

the

recent

FDA

approved

drug,

Fulyzaq

is

the

first

anti-diarrheal

drug

for

adult

HIV/AIDS

patients

derived

from

the

plant

Croton

lechleri.

Also,

of

currently

known

anti-TB

drugs

eight

drugs

i.e.,

Rifampicin,

Rifapentine,

Streptomycin,

Kanamycin,

Amikacin,

Capreomycin,

Viomycin

and

Cycloserine

have

been

derived

from

microbial

sources

(NPs)

(see

Additional

file

Table

Many

experimentally

tested

anti-mycobacterial

NPs

are

scattered

throughout

scientific

literature

In

order

to

systematically

evaluate

their

anti-mycobacterial

properties,

this

data

needs

to

be

systematically

collated

and

analyzed.

A

number

of

pathway

and

cheminformatics

tools

for

TB

drug

discovery

have

been

developed

but

are

not

exclusively

for

plant

derived

anti-mycobacterial

compounds

At

present,

Universal

Natural

Products

Database

(UNPD)

is

the

largest

freely

available

resource

for

NPs

and

holds

a

total

of

NPs

of

plants,

animals

and

microbial

origin

The

various

other

NP

databases

and

chemical

libraries

freely

available

for

public

use

are

NuBBE

CamMedNP

SuperNatural

HIT

NPACT

TCM

Database@Taiwan

TCMID

HIM-herbal

ingredients

in-vivo

metabolism

database

AfroDB

and

ConMedNP

to

name

a

few.

However,

information

related

to

phytomolecules

and

plant

extracts

against

mycobacterial

strains

is

very

limited.

Moreover,

such

data

resources

should

be

equipped

with

analysis

tools

to

prioritize

compounds

for

early

stage

drug

discovery,

which

is

missing.

Furthermore,

all

the

resources

may

follow

a

standardized

global

ontology

so

that

the

data

shared

by

individuals

or

laboratories

is

interoperable

and

facilitate

implementation

of

the

semantic

web

features..

BioPhytMol

has

been

developed

as

a

database

cum

drug

discovery

platform

that

can

be

used

to

prioritize

drug-like

compounds.

A

number

of

experimentally

verified

antimycobacterial

phytomolecules

along

with

their

experimental

assay

preparation

and

bioactivity

details

have

been

manually

compiled

from

literature.

Tools

providing

text

searching,

browsing

and

chemical

similarity

searching

for

phytomolecules

have

been

integrated

to

utilize

BioPhytMol

as

a

drug

discovery

platform.

The

chemical

structure

similarity

tool

can

be

exploited

to

perform

exact,

substructure,

superstructure

and

perfect

search

against

various

drug-like

chemical

libraries.

This

facility

may

be

helpful

to

assign

chemical

classes

and

bioactivity

to

novel

compounds.

Users

can

also

generate

analogs

for

their

target

specific

inhibitors.

As

of

now

the

resource

holds

a

total

of

entries

including

plant

families

(comprised

of

genera

and

species)

from

global

flora.

In

total,

purified

compounds

have

been

curated

against

different

target

mycobacteria.

These

are

reported

along

with

their

biological

activity

and

experimental

details

as

described

below.

The

resource

is

available

at

res.in/biophytmol/.

B O B I O O O B O O O O O B B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B B I B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O B O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B B I O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


TEFIS: A single access point for conducting multifaceted experiments on heterogeneous test facilities A few years ago, an experimental facility composed of networking gear and simulation tools was sufficient for testing the main features of a prototype before the final product could be launched to the Internet market. This paradigm has certainly changed, but the lack of platforms enabling the realistic assessment of the different facets of a product, including cross-cutting trials across different testbeds, poses strong limitations for researchers and developers. In light of this, we present an open platform that offers a versatile combination of heterogeneous experimental facilities called ''TEstbed for Future Internet Services'' (TEFIS). TEFIS provides a single access point for conducting cutting-edge experiments on testbeds that supply different capabilities, including testbeds dedicated to network performance, software performance, grid computing, and living labs. We shall show that TEFIS covers the entire life-cycle of a multifaceted experiment, with the advantage that a single testrun can seamlessly execute across different experimental facilities. In order to demonstrate the potential and versatility of the TEFIS platform, we describe the deployment of four distinct experiments and provide a set of results highlighting the benefits of using TEFIS. The experiments described in this article cover: (i) the experimentation with an open API called OPENER (which is an open and programmable environment for managing experimentation with SDN applications); (ii) an application for skiers and tourists at the Megève ski resort in France; (iii) an application that can dynamically adapt the Quality of Experience (QoE) of multimedia services for mobile users; and (iv) an augmented reality workspace for remote education and learning purposes based on videoconferencing. The heterogeneity of applications, protocols, and devices in today's Internet has dramatically increased the complexity and cost of thoroughly testing new technologies, especially in scenarios that aim to be as close as possible to the target ones. Indeed, the success of a new Internet technology now depends both on objective and subjective aspects, so traditional simulation and experimentation strategies are being relegated to the very early stages of prototype testing. In this context, one of the main challenges faced by researchers and developers is the lack of experimental platforms supporting the assessment of the different facets of a service. For instance, consider the case of an experimenter who would like to assess not only the performance of a distributed application along with the experience obtained by the users, but also the quality and reliability of the software packages that support the application itself. Ideally, experimenters would like to run such tests in ''a single place'', irrespective of how many different test facilities are required, and they would like to be able to manage their experiments throughout its entire life-cycle. Moreover, experimenters would also like to be able to build upon other experimenters' work, as well as learn from their experiences. This would obviously facilitate the overall experimentation process, and would allow experimenters to contribute with their own work to the existing knowledge base.. Our aim in this paper is to present a new platform with the potential to meet these ideals. The ''TEstbed for Future Internet Services'' (TEFIS) [1] is such a platform and represents a large European effort enabling multifaceted experiments on different test facilities. More specifically, TEFIS offers a versatile combination of heterogeneous experimental facilities through ''a single access point'', and it is built as an open platform that currently provides seamless access for testing services on six different testbeds, namely: (1) PACAGrid [2] (a grid computing facility for experimenting with computationally intensive applications); (2) ETICS [3] (an e-infrastructure that provides support for testing the entire software development lifecycle); (3) SQS-IMS [4] (an infrastructure for testing protocols and mobile applications over the IP Multimedia Subsystem (IMS) [5] ); (4) The Botnia Living Lab [6] (a Living Lab that offers end-user engagement in the design and testing of Future Internet technologies and services); (5) Kyatera [7] (a high-speed optical network for supporting experimentation of technologies with large bandwidth demands); and (6) PlanetLab [8] (a testbed for distributed systems and network research on a large scale over the Internet).. By seamlessly combining these different testbeds, TEFIS can cover most of the development and testing activities, such as building and packaging software, system integration, Service-Level Agreement (SLA) dimensioning, largescale deployment and testing, compliance testing, as well as ''user-level evaluation'' of run-time services. As we shall show, the TEFIS platform provides all the necessary services enabling the management of underlying resources for executing complex experiments on multiple test facilities. In particular, TEFIS handles aspects such as resource management (e.g., resource access, matching and identification of resources that can be activated, etc.), software deployment, measurement-related services for a variety of testbeds, as well as data persistence. One of the key advantages of TEFIS is that this handling is not exposed to experimenters, since the platform abstracts the internals of the different testbeds by means of a flexible interface that enables not only the robust configuration of experiments, but also the automation and repeatability of the same.. Overall, this paper makes the following contributions. We describe an open architecture that is sufficiently general as to enable multifaceted experimentation on multiple (and quite different) testbeds. We show that the combination of testbeds is made possible by means of ''connectors'', which provide the desired level of abstraction for experimenters while facilitating the process of adding and/or removing testbeds as needed. As a proof-of-concept, we describe four distinct experiments that were recently run on TEFIS, and show how they have benefit from the resources offered by the different test facilities. We also examine some of the future challenges faced by experimental platforms such as TEFIS, including a discussion on business and sustainability models allowing the continuity of these platforms.. We contend that the TEFIS model can be positioned as an enabler for building a community of experimenters, which can increase their expertise and share their knowledge in order to accelerate, and improve, the design and evaluation phases for a broad spectrum of Internet technologies.. The remainder of this paper is organized as follows. In Section 2, we review related work. In Section 3, we describe in more detail the experimental facilities available through TEFIS and the complementarities and synergies that they bring for experimenters. Later, in Section 4, we outline the architecture and the main components of the TEFIS platform. A step-by-step procedure for creating a new experiment as well as the internal workflow for running a multifaceted experiment through TEFIS are described in Section 5. In Section 6, we depict a set of four different use cases, which perfectly capture the potential and the capabilities of the platform for supporting the inherent heterogeneity of multifaceted testing. Finally, Section 7 discusses the main challenges for maintaining experimental platforms like the one presented in this work, and concludes the paper. During the last few years, several initiatives across the world have focused on providing support and resources for cutting-edge experimentation under the framework of the so called ''Future Internet''. In this section, we review some of the most prominent endeavors in this direction, and highlight the features that are unique in TEFIS. Clearly, we cannot cover here all the initiatives in the area, so we will mainly address a subset of them with special focus on some of the activities recently started in Europe, in the US, and Japan.. In reference to Europe, the advances are mainly driven by a research program launched and managed by the European Commission called the ''Future Internet Research and Experimentation'' (FIRE) initiative [9] . FIRE seeks to foster the development of experimental facilities capable of supporting the testing of a broad spectrum of ICT technologies, protocols, and applications in Europe. TEFIS is in fact just one of the experimental platforms that are being developed under the FIRE umbrella. In a nutshell, the initiatives in FIRE whose objectives are closer to TEFIS can be categorized into three groups, namely: (i) those addressing Data and Service Management experimentation (including clouds); (ii) those addressing experimentation with Networking technologies and protocols; and (iii) the ones developing platforms for experimenting in the area of Content-Centric Networking. It is worth noting that these are indeed overlapping groups, since, for instance, some of the activities in the area of ''Content-Centric Networking'' also address ''Data and Service Management'' issues and vice versa.. In the first group, we found initiatives such as NOVI [10], BonFIRE [11] , and LAWA [12] . NOVI is exploring ways to compose and manage virtualized infrastructures in the form of ''baskets'' of virtual resources and services that are made available through a testbed federation. With a different approach, BonFIRE targets the development of a platform for innovative experimentation on multi-site cloud facilities. LAWA, on the other hand, is mainly focused on developing an experimental testbed for large-scale data analytics-a kind of virtual web observatory-and is also based on a federation of distributed FIRE facilities.. The second group brings together initiatives such as OpenLab [13] and OFELIA [14] , as well as a number of already concluded activities, such as PanLab [15] , OneLab [16] and OneLab2 [17] , and FEDERICA [18] . The portfolio of testbeds offered in OpenLab is rich and diverse and includes: PlanetLab Europe (PLE) [19] ; the European Traffic Observatory Measurement Infrastructure (ETOMIC) [20] which offers high-precision network measurements; the Network Implementation Testbed using Open Source code (NITOS) [21] , along with other experimental facilities for testing applications and technologies in the areas of radio, networking, IMS, and data centers [13] . With a different target, OFELIA [14] is one of the initiatives trying to create a flagship test facility allowing experimentation with OpenFlow [22] in Europe. In this second group, we also found the PanLab initiative, which addressed the need for large-scale testing facilities in the areas of telecommunications and information technologies. PanLab implemented an infrastructure for federating testbeds based on the ''Teagle'' portal [23] . Teagle offers a central coordination instance allowing experimenters to express their testing needs, while also allowing testbed providers to browse, configure, deploy, and register new resources to be provided by the testbed federation. It is worth highlighting that, due to its numerous advantages, the Teagle portal is also being used in our own platform. In addition, OneLab and OneLab2 offered access to four testbeds-three of which were already mentioned-namely, PlanetLab Europe, ETOMIC, NITOS, plus the DIMES topology measurement infrastructure [24] . Last, but not least, FEDERICA focused on exploring ways of virtualizing and slicing an experimental network infrastructure, with the aim of delivering solutions for managing, controlling, and monitoring a set of parallel virtual networks.. In the third group, the initiative that is developing an experimental platform that shares part of our vision in the subject of living labs is EXPERIMEDIA [25] . The latter is developing a test facility especially devised for largescale future media Internet experiments, and it includes the following testbeds: the Schladming Ski Resort [26] , the Multi-Sport High Performance Center of Catalonia [27] , the Foundation for the Hellenic World [28] , and the 3D Innovation Living Lab [29] .. All the initiatives cited thus far run under the FIRE umbrella [9] , targeting different aspects around the experimentation challenges in the Future Internet. These initiatives are in one way or another making their mark on the subject, but, as mentioned above, they are not the only ones that are developing platforms for the experimental research sector in ICT in Europe. The flagship initiative of Future Internet Public Private Partnership (PPP) Programme, FI-WARE [30] , goes beyond experimental research objectives and targets Future Internet professional services as well. It plans to build a Core Platform for the Future Internet, traversing way beyond the technical scopes of the three groups of FIRE initiatives, mentioned earlier.. Among the several initiatives working towards developing a platform for Smart Cities, FI-WARE aims at becoming the open platform for Smart Cities in Europe.. In the US, the flagship initiative is the well-known ''Global Environment for Network Innovations'' (GENI) [31] . Unlike Europe, GENI is particularly focused on clean-slate research on networking, and to this end, US researchers have adopted a bottom-up approach. GENI is exploring networks of the future through the creation of a virtual Internet-scale laboratory, and, among the many activities carried out under the GENI umbrella, such as, INSTAGENI [32] and K-GENI [33] , lies the generation of specifications for testbed management.. It is worth mentioning that similar initiatives are also being developed in Brazil, China, and particularly, in Japan. Among the most important ones in Japan are the activities undertaken by the National Institute of Information and Communications Technology (NICT) [34] . NICT is currently driving the development of the next generation of its national testbed, i.e., the Japanese Gigabit Network (JGN). This effort is being carried out under the framework of the NICT project called JGN eXtreme (JGN-X) [35] . As is the case with GENI in the US, the JGN-X initiative is also strongly focused on developing a platform for testing new networking paradigms, new protocols, new control and management features, as well as network virtualization mechanisms.. In this scenario, we consider that TEFIS has the potential to bring the experimentation processes one step further. Even though all the experimental platforms described above can handle the entire life-cycle for their corresponding experiments, TEFIS promises to add value by covering not only the testing phase during the software development life-cycle, but also the user-level of experience and satisfaction for a broad spectrum of Internet technologies and services. Indeed, along with FIRE, the European Network of Living Labs [36] provides an alternative, and complementary approach to Future Internet experimentation. A Living Lab allows for the testing of aspects which might not have necessarily been considered by the developers of the technology. Such aspects often include user motivation and real-world environment, identifying practical and additional demands for the systems and the services running on them, etc. As shown in Table 1 , by combining test facilities that can offer the assessment of the quality and interoperability of software, with FIRE experimental facilities and Living Labs, our platform provides a powerful environment for conducting multifaceted experiments on a broad range of Internet services.. The latest trend in testing and experimentation facilities in Europe shows a keen interest in the concept of federation of cross-technology testbeds for providing a single platform to foster cross-domain innovative experiments for Future Internet. FED4FIRE [37] is one step in that direction which targets to make the numerous diverse initiatives of the FIRE [9] programme, mentioned earlier, reliably accessible through one platform. This way a federation of testbeds, apart from broadening the horizons for many possible and innovative experiments, also facilitates the path to make testbeds sustainable. The TEFIS ideology is very much aligned with this direction as it integrates different experimental facilities. We now proceed to describe in more detail the features of the experimental facilities that are currently accessible through TEFIS.

O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O


Variational data assimilation using targetted random walks The variational approach to data assimilation is a widely used methodology for both online prediction and for reanalysis. In either of these scenarios, it can be important to assess uncertainties in the assimilated state. Ideally, it is desirable to have complete information concerning the Bayesian posterior distribution for unknown state given data. We show that complete computational probing of this posterior distribution is now within the reach in the offline situation. We introduce a Markov chain-Monte Carlo (MCMC) method which enables us to directly sample from the Bayesian posterior distribution on the unknown functions of interest given observations. Since we are aware that these methods are currently too computationally expensive to consider using in an online filtering scenario, we frame this in the context of offline reanalysis. Using a simple random walk-type MCMC method, we are able to characterize the posterior distribution using only evaluations of the forward model of the problem, and of the model and data mismatch. No adjoint model is required for the method we use; however, more sophisticated MCMC methods are available which exploit derivative information. For simplicity of exposition, we consider the problem of assimilating data, either Eulerian or Lagrangian, into a low Reynolds number flow in a two-dimensional periodic geometry. We will show that in many cases it is possible to recover the initial condition and model error (which we describe as unknown forcing to the model) from data, and that with increasing amounts of informative data, the uncertainty in our estimations reduces. Data assimilation, using either filtering [1, 2] or variational approaches [3] [4] [5] , is now a standard component of computational modelling in the geophysical sciences. It presents a major challenge to computational science as the space-time distributed nature of the data makes it an intrinsically fourdimensional problem. Filtering methods break this dimensionality by seeking sequential updates of three-dimensional problems and are hence very desirable when online prediction is required [6] [7] [8] . However, when reanalysis (hindcasting) is needed, for example to facilitate parameter estimation in sub-grid scale models, it is natural to confront the fully four-dimensional nature of the problem and not to impose a direction of time; variational methods are then natural [9] [10] [11] .. However, for both filtering and variational methods an outstanding computational challenge is the incorporation of uncertainty into the state estimation. Many of the practical methods used to confront this issue involve ad hoc and uncontrolled approximations, such as the ensemble Kalman filter [12] . While this has led to a very efficient methodology, and will doubtless continue to have 404 S. L. COTTER, M. DASHTI AND A. M. STUART significant impact for some time to come, there remains scope for developing new methods which, while more computationally demanding, attack the problem of quantifying statistical uncertainty without making ad hoc or uncontrolled approximations. The natural setting in which to undertake the development of such new methods is the Bayesian framework in which one seeks to characterize the posterior distribution of state given data [13] . The Markov chain-Monte Carlo (MCMC) methodology provides a well-founded approach to fully probing this posterior distribution. The purpose of this paper is to show that sampling of this posterior distribution is now starting to fall within reach, via MCMC methods, in the offline situation. We also emphasize the possibility of, in future work, using such MCMC based studies of the posterior distribution to benchmark more practical algorithms such as filtering and variational approaches, both of which may be viewed as providing approximations to the posterior.. The Bayesian approach to inverse problems [14] is a conceptually attractive approach to regularization which simultaneously forces consideration of a number of important modelling issues such as the precise specification of prior assumptions and the description of observational noise [15] . However, in the context of PDE inverse problems for functions the approach leads to a significant computational difficulty, namely probing a probability measure on the function space. To be clear: the probabilistic viewpoint adds a further degree of high dimensionality to the problem, over and above that stemming from the need to represent the unknown function itself. Variational methods, which minimize a functional which is a sum of terms measuring both model-data mismatch and prior information, corresponds to finding the state of maximal posterior probability, known as maximum a posteriori (MAP) estimator in statistics [15] . This will be a successful approach if the posterior distribution is in some sense 'close' to Gaussian and with small variance. However, since the dynamics inherent within the observation operator can be highly non-linear, and since the observations may be sparse, it is not necessarily the case that a Gaussian approximation is valid; even if it is, then the spread around the MAP estimator (the variance) may be important and detailed information about it is required. Characterizing the whole posterior distribution can be important, then, as it quantifies the uncertainty inherent in state estimation.. Markov chain-Monte Carlo (MCMC) methods are a highly flexible family of algorithms for sampling probability distributions in high dimensions [16, 17] . There exists substantial literature on the use of the MCMC methodology for the solution of inverse problems from fluid mechanics [18] [19] [20] [21] [22] [23] and from other application domains [24] [25] [26] [27] [28] . A key computational innovation in this paper, which distinguishes the methods from those in the preceding references, is that the algorithms we use estimate the posterior distribution on a function space in a manner which is robust to mesh refinement.. We employ random walk Metropolis-type algorithms, introduced in [29] , and generalized to the high-dimensional setting in [30] [31] [32] . The method that we implement here does not use any gradient information (adjoint solvers) for the model-data mismatch. It proposes states using a random walk on the function space and employs a random accept/reject mechanism guided by the model-data mismatch on the proposed state. This method is hence straightforward to implement, but not the most efficient method: other methods (Langevin or Hybrid Monte-Carlo), which require implementation of an adjoint model, can explore the state space in a more efficient manner [32] . In this paper, we stick to the simpler Random Walk method as a proof of concept, but the reader should be aware that gradient methods, such as the Langevin Algorithm, or Hybrid Monte-Carlo, could be implemented to increase the efficiency. The basic mathematical framework within which we work is that described in [33] in which we formulate Bayesian inverse problems on the function space, and then study data assimilation problems arising in fluid mechanics from this point of view. The abstract framework of [33] also provides a natural setting for the design of random walk-type MCMC methods, appropriate to the function space setting. The probabilistic approximation theory associated to such algorithms is studied in [34] .. In Section 2, we describe the data assimilation problems that we use to illustrate our Bayesian MCMC approach; these problems involve the incorporation of data (Eulerian or Lagrangian) into a model for Stokes' flow. Section 3 briefly describes the mathematical setting for our data assimilation problems, and for the MCMC algorithm that we employ throughout the paper. In 405 Sections 4 and 5, we consider Eulerian and Lagrangian observations, respectively, incorporating them into a two-dimensional Stokes' flow model with periodic boundary conditions. In each case we will consider first the problem of recovering the initial condition of the velocity field, given that we assume that we know the forcing present in the system during the window of observation. We will then consider, in each of the two data scenarios, what happens if we assume that we know the forcing inherent in the system and incorporate this into the statistical algorithm, but in fact there is a degree of model error which can only be accounted for by altering the distribution on the initial condition. Finally, we will consider full initial condition and model error inference (which together are equivalent to finding a probability distribution on the state of the velocity field for the duration of the window of observation). In particular, we study the extent to which model error may be recovered from data, and the uncertainty in this process. In Section 6, we present some brief conclusions. We have argued that, in situations where uncertainty quantification is important, the use of an MCMC method to fully explore the posterior distribution of state given data is both worthwhile and increasingly viable. We have investigated in some details the properties of the posterior measure for simplified models of data assimilation in fluid mechanics, for both Eulerian and Lagrangian data types, with and without model error. Through this work, we were able to say more about what kind of information we can garner from these different data types in different scenarios. Although only indicative, being a markedly simpler model than any model that is currently used in practical meteorological and oceanographical scenarios, it still gives us a better idea of how to attack these problems, and of how much information is contained within different types of noisy observation in them. Furthermore, in the current computational practise various simplifications are used-primarily filtering or variational methods [33] -and the posterior measure that we compute in this paper could be viewed as an 'ideal solution' against which these simpler and more practical methods can be compared. To that end it would be of interest to apply the ideas in this paper to a number of simple model problems from data assimilation in meteorology or oceanography and to compare the results from filtering and variational methods with the ideal solution found from MCMC methods.. We have also shown that in the limit of a large amount of informative data, whether that be created by increasing the number of observation times, or by increasing the number of tracers being observed, or the number of observation stations in the Eulerian case that the posterior distribution converges to an increasingly sharply peaked measure on the field/forcing function that created the data, with and without model error. Proving such a result presents an interesting mathematical challenge that will give insight into the large data scenario.. We have also seen certain scenarios in which the random walk method was pushed up to, and beyond its boundaries of efficacy. Implementation of gradient-based methods such as the (Metropolis-Adjusted Langevin) MALA algorithm, as shown in [33] , is also of great interest, and may be useful in tackling these more complex posterior densities.. There are also many other application domains to which the algorithmic methodology developed here could be applied and another future direction will be to undertake similar studies to those here for problems arising in other application areas such as subsurface geophysics and applications in image processing such as shape registration.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O