Neural network learning of optimal Kalman prediction and control Although there are many neural network (NN) algorithms for prediction and for control, and although methods for optimal estimation (including filtering and prediction) and for optimal control in linear systems were provided by Kalman in 1960 (with nonlinear extensions since then), there has been, to my knowledge, no NN algorithm that learns either Kalman prediction or Kalman control (apart from the special case of stationary control). Here we show how optimal Kalman prediction and control (KPC), as well as system identification, can be learned and executed by a recurrent neural network composed of linear-response nodes, using as input only a stream of noisy measurement data.The requirements of KPC appear to impose significant constraints on the allowed NN circuitry and signal flows. The NN architecture implied by these constraints bears certain resemblances to the local-circuit architecture of mammalian cerebral cortex. We discuss these resemblances, as well as caveats that limit our current ability to draw inferences for biological function. It has been suggested that the local cortical circuit (LCC) architecture may perform core functions (as yet unknown) that underlie sensory, motor, and other cortical processing. It is reasonable to conjecture that such functions may include prediction, the estimation or inference of missing or noisy sensory data, and the goal-driven generation of control signals. The resemblances found between the KPC NN architecture and that of the LCC are consistent with this conjecture. The Kalman optimal filter and controller (Kalman, 1960) are classical solutions for efficient optimal estimation (which includes prediction, filtering, and smoothing) and optimal control in linear systems. They also form the basis for extensions that yield approximately optimal solutions for certain types of nonlinear systems. Within the field of neural networks, a great many algorithms for prediction and control in a variety of settings have been developed. Yet there exists, to my knowledge, no neural algorithm for learning the optimal Kalman filter (KF), nor for learning the optimal Kalman controller (KC) except in the stationary case (discussed in section 5.2).. In addition, the classical Kalman algorithms assume that the parameters characterizing the external system (the 'plant') and the measurement process are known in advance. When they are not known, a separate process of system identification is typically performed.. In this paper we derive a neural network (NN) circuit and algorithm that learns and executes Kalman estimation and control, and that also determines the required combinations of plant and measurement process parameters, using as input only a stream of noisy measurement data vectors (and, for the controller, the specification of the cost function whose value is to be optimized). The differences between the Kalman filter and controller learned by the network, and those derived using the classical Kalman algorithms, can be made arbitrarily small, provided that certain expectation values over distributions are sufficiently well approximated by the corresponding finite-sample statistics 1 (as discussed in Sections 3 and 4).. The resulting artificial neural circuit and algorithm may prove useful for implementing the learning and execution of Kalman prediction and control, and its nonlinear extensions, in parallel systems consisting of simple processors.. The resulting circuit architecture also has distinctive features that invite comparison with aspects of biological neural networks, particularly in cerebral cortex, and may help in exploring the possible functions of such networks.. The paper is organized as follows. Section 2 summarizes the optimal linear estimation and control problems, and the classical Kalman filter and controller algorithms. In section 3 we derive a neural network algorithm that both solves the system identification problem -i.e., learns the dynamical properties of the plant (as these properties are reflected in the measurement data) -and learns the optimal Kalman filter with arbitrary accuracy. The derivation proceeds in several stages. We first transform the classical Kalman estimation equations into a form that explicitly involves only the input measurement vectors, and not the plant state vectors themselves. (We do this because the plant state is unknowable from the data in principle, even apart from noise, when, as we assume, the transformation from plant state to measurement vector is not specified to the NN.) We then derive a learning procedure that exactly implements these transformed equations, but that is expressed in terms of certain expectation values. Next, we derive a neural network algorithm that implements this learning procedure with arbitrary accuracy (depending upon how well the expectation values are approximated by sample statistics). Some distinctive elements of this algorithm include: (a) the use of local neural network methods to perform the required learning or use of the inverse of an error covariance matrix; (b) the generation of this covariance matrix by using either an ensemble of measurement vectors at each time step (e.g., the positions of a set of tracked features in a visual scene), a single vector tracked over time to generate such an ensemble, or a combination of the two; (c) the simultaneous learning of the Kalman filter, use of that filter to predict the future plant state, and refinement of the learning of the plant dynamical parameters; and (d) a specific recurrent circuit architecture, and sequencing of computations, that are implied by the algorithm. Finally, the joint NN learning of the plant dynamics and the Kalman filter is illustrated with a numerical example.. In section 4, we derive a neural algorithm for Kalman control. The several stages of the derivation are similar to those used for Kalman estimation. There are evident similarities that result from the mathematical duality (Kalman, 1960 ) between Kalman's optimal estimation and optimal control solutions, but there is an additional distinctive feature: Kalman's duality includes a time reversal operation, so that the Kalman control matrix is computed by a process that operates 'backward in time,' from the future time of target (goal) acquisition to the present. We show how this requirement is implemented within the neural algorithm, which handles a decrementing time index during the learning process, and generates a sequence of controller outputs to the plant in physical (forward-moving) time. We then integrate the control method into the same neural circuit and algorithm that handles estimation and system identification.. In section 5 we discuss several issues. First, we identify ways in which the computational task -Kalman prediction and control -places constraints on the type of NN circuitry and signal flows that are involved in performing that task. Second, we comment on applications to artificial NN designs, and discuss prior work that has used NNs in conjunction with Kalman methods.. Finally, in subsection 5.3 and the speculative subsection 5.4, we identify certain resemblances between the artificial NN that we are led to by the Kalman prediction and control (KPC) constraints, and the architecture (and proposed signal flows) of the putative 'local cortical circuit' (LCC, minicolumn, canonical microcircuit) of mammalian cerebral cortex. The resemblances between the KPC NN and the LCC, and important caveats that apply to the interpretation of these resemblances, are discussed.. Section 6 summarizes and concludes the paper. The approach taken here has been to (a) pose a well-defined computational task -Kalman prediction and control -that is a prototype of the more general and powerful prediction and control processes that are likely to be important in cortex; (b) select a simple typical set of allowed NN operations, rather than invoking more complex NN dynamics specially tailored to the task; (c) see whether a NN algorithm can be devised that does not compromise, change, or limit the computational task; (d) see what constraints that task imposes on the NN's circuitry and signal flows; and (e) compare the resulting NN circuitry with that of the biological system of interest.. We have shown how optimal Kalman estimation and control, and system identification, can be learned and executed by a neural network having simple computational elements. In progressing from the classical Kalman equations to a NN algorithm, we have find that the computational task appears to impose significant constraints on the resulting NN architecture, circuitry, and signal flows.. When we compare the resulting architecture to that of recurrent neural circuits found in brain, and to LCC architecture in particular, we find certain resemblances. The LCC architecture has been suggested to perform core functions (as yet unknown) that underlie sensory and motor processing in general. It is plausible that such functions may include prediction, the estimation or inference of missing or noisy sensory data, and the goal-driven generation of control signals. Thus the resemblances found between the KPC NN architecture and that of the 'local cortical circuit' may not be coincidental.. However, before one can infer from such resemblances that the LCC is likely to be performing a particular set of core functions, much more evidence is required. The present work fits into a broader context of ongoing NN research by many workers, in which (a) a range of biologically realistic neural coding strategies (e.g., population or spike coding) is being studied, to see how simple neural computations can be carried out using these codes, and (b) a range of computational tasks in prediction and control (e.g., Bayesian inference) is being explored, to see how recurrent NNs may implement such tasks. From the experimental side, further elucidation of both the functional connectivity and signal flows in local cortical circuitry is of course also required. It will be interesting to see whether certain types of computational tasks are found to be associated with specific architectural features in NNs and, if so, whether such mappings have useful implications for understanding biological neural function.. Such continuing interaction between theory and experiment may not only contribute to elucidating core aspects of cortical function, but may also lead to insights into new methods for nonlinear estimation and control.. (2) The plant and measurement equations for x t+1 and y t yield y t+1 =F Hx t +ũ t + Hm t + n t+1 ;. thus. Since (a) the noise terms m t , n t , and n t+1 are mutually independent and have zero mean; (b) η t depends on n t (through y t ) but not on m t or n t+1 ; (c) R and Z are symmetric matrices; and (d) E(m t m t ) = Q, E(n t n t ) = E(n t+1 n t+1 ) = R, E(η t n t ) = −E(n t n t ) = −R, and E(η t η t ) = Z t ; we obtain. which equals Z t+1 by Eq. 9. We have shown how to perform the learning and execution of Kalman estimation and control, as well as system identification, using a neural network. The method is asymptotically exact in the limit that certain sample averages of computed quantities approach the corresponding expectation values over the distributions of those quantities. The matrices Z and T that are iteratively computed, in order to learn and execute KF and KC respectively, are each equal to the covariance of a distribution of computed activity vectors. In each case, vectors evolve over time via a sequence of transformations performed by the NN, and are used, via Hebbian learning, to update a matrix of connection weights that represents the KF or the KC matrix. The logical path of the derivation proceeds from the classical Kalman solutions, to a transformed set of equations that involves only quantities measured by the NN, to a set of signal flows and computations, and finally to a layered NN architecture and circuitry that supports those computations.. Both the signal flows and the NN architecture appear to be constrained by the requirements of the Kalman neural algorithm (apart from small variations); that is, they do not appear to represent merely one among a large number of disparate possible choices of architecture or signal flow.. In the remainder of this section, we discuss. (1) the considerations that constrain the architecture and signal flows; (2) applications to artificial NN design, and prior work; and (3) resemblances between the derived architecture and biological networks in neocortex, and caveats involved in drawing inferences regarding biological function.

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Design

and

implementation

of

real

time

wideband

channel

simulator

This

paper

describes

the

design

and

implementation

of

a

DSP

based

real

time

wideband

channel

simulator.

The

simulator

implementation

uses

a

floating

point

and

a

fixed

point

DSPs.The

simulator

has

taps

and

baseband

bandwidth

of

MHz.

It

has

flexibility

to

generate

several

channel

models

under

varied

environmental

conditions.

To

validate

the

functionality

of

the

simulator,

the

baseband

data

is

applied

to

the

simulator

input

and

its

output

is

statistically

analyzed

and

the

results

are

compared

with

those

predicted

analytically.

Wireless

communications

are

inherently

unreliable

due

to

their

time

varying

nature,

multipath

propagations

and

presence

of

interference

signals

from

other

users.

In

the

presence

of

these

impairments,

substantially

higher

power

must

be

transmitted

to

overcome

these

impairments

inorder

to

acheive

acceptable

symbol

error

rate

in

any

kind

of

radio

channel.

To

design

reliable

and

efficient

wireless

systems,

it

is

essential

to

understand

the

behavior

of

radio

channels

in

different

environments..

Statistical

channel

modeling

plays

a

vital

role

in

the

design

of

reliable

wireless

communication

system.

Channel

modeling

is

the

first

step

towards

the

efficient

wireless

system

design.

The

purpose

of

channel

modeling

is

to

compute

and

estimate

the

various

first

and

higher

order

statistical

parameters

of

the

fading

channel.

These

parameters

include

Doppler

spread,

the

time

constants

of

fading,

average

fade

duration,

level

crossing

rates,

amplitude

probability

distribution

functions

and

the

coherence

bandwidth.

For

this

purpose,

measurements

have

been

taken

in

different

environment

to

characterize

the

channel..

Over

the

past

few

decades,

a

number

of

experiments

have

been

performed

to

characterize

mobile

channels

in

urban,

suburban,

mountainous,

wooded

and

highway

environments

On

the

basis

of

these

measurements,

*Correspondence:

miakram@kfupm.edu.sa

Department

of

Electrical

Engineering,

King

Fahd

University

of

Petroleum

Minerals,

Dhahran,

Saudi

Arabia

several

channel

models

have

been

proposed

to

explain

the

observed

statistical

nature

of

the

fading

channels

between

fixed

base

stations

and

mobile

stations.

These

include

short

term

fading

models

like

the

well-known

Rayleigh,

Rice

Hoyt

Nakagami-m

and

Weibull

and

for

longer

term

lognormal

model

has

been

used

To

evaluate

the

design

and

performance

of

a

communication

system,

it

is

desirable

to

evaluate

it

in

realistic

situations.

The

experiments

can

be

performed

directly

in

a

vehicle,

driving

through

different

environments.

However,

this

is

a

time-consuming

and

expensive

task

and

it

requires

the

presence

of

measurement

equipments

with

proper

calibrations.

Moreover,

the

field

trials

can

be

affected

by

unintended

uncontrollable

circumstances.

The

inexpensive

and

flexible

option

is

to

use

a

real

time

channel

emulator

and

measure

the

performance

in

a

laboratory

environments

as

in

The

commercially

available

channel

emulators

available

may

not

offer

the

user

enough

flexibility

when

configuring

the

wireless

channel

parameters

to

test

the

system

under

different

environmental

conditions.

Such

simulators

do

not

cover

scenarios

under

different

channel

models

like

Hoyt,

Rice

Hoyt

etc.

A

low

cost

channel

simulator

is

therefore

required

that

models

different

scenarios

and

at

the

same

time

provide

the

user

flexibility

to

measure

the

performance

of

the

wireless

transceiver

under

environmental

conditions..

Over

the

past

few

decades,

efforts

have

resulted

in

several

designs

and

implementation

of

real

time

simulators.

Early

efforts

were

based

on

analog

components

The

development

of

real

time

simulator

starts

in

when

developed

the

first

Rayleigh

based

channel

simulator.

The

simulator

used

Zener

diode

to

generate

Gaussian

random

variable.

But

with

the

advent

of

digital

computers,

micro-controllers,

fast

Analog

to

Digital

Converters

(ADCs)

and

Digital

to

Analog

Converters

(DACs),

the

analog

components

were

replaced

by

digital

thereby

increasing

the

reliability

and

flexibility

of

simulators.

Comroe

et

el

first

used

discrete

digital

logic

in

its

simulator.

With

the

development

of

High

Speed

Digital

Signal

Processors

(DSPs),

the

DSP

based

simulators

were

developed.

Goubran

et

al

used

bit

fixed

point

DSP

for

implementation

and

simulated

the

Gaussian

quadrature

components

along

with

the

log-normally

distributed

Line

of

Sight

(LOS)

component.

Turkmani

et

al

used

DSP

chip

for

the

development

of

a

narrow-band

simulator.

Cullen

et

al

reported

a

frequency

selective

simulator

using

DSP

and

integrated

circuits.

It

had

a

baseband

bandwidth

of

MHz

and

maximum

Doppler

frequency

of

Hz.

Chen

et

al

used

DSP

to

design

a

frequency

selective

simulator.

Salkintzis

et

al

developed

taps

wide-band

channel

simulator

having

maximum

signal

bandwidth

of

MHz.

It

used

two

bit

DSP

floating

point

processors.

Papenfuss

et

al

used

a

hybrid

DSP

FPGA

architecture

to

build

a

wide-band

channel

simulator.

It

was

capable

of

simulating

delay

taps

and

had

a

baseband

bandwidth

of

MHz.

Satellite

Channel

simulator

has

been

developed

in

using

DSP

platform.

Kominakis

et

al

developed

a

narrow-band

fast

and

accurate

simulator.

Khars

et

al

developed

a

MHz

taps

wide-band

simulator

using

DSPs

for

each

tap)

for

the

generation

of

complex

coefficients.

A

narrow-band

DSP

based

channel

simulator

has

also

been

developed

in

Over

the

last

decade,

the

use

of

Field

Programmable

Gate

Arrays

(FPGAs)

in

DSP

applications

has

become

quite

common.

The

FPGA

based

simulators

have

also

been

developed

and

their

implementations

have

been

described

in

One

important

application

of

wireless

communication

is

Vehicle

to

Vehicle

communications

where

both

the

transmitter

and

receiver

are

in

motion.

The

communication

finds

its

applications

in

mobile

ad-hoc

wireless

networks,

intelligent

highway

systems,

emergency,

military

and

security

vehicles.

The

implementation

of

communications

enhances

road

safety

due

to

reduction

in

the

number

of

accidents,

improvement

in

highway

traffic

flow

efficiency

and

real

time

data

sharing

without

involving

the

cellular

network

which

leads

to

efficient

fuel

consumption

and

reduced

travel

time.

The

statistical

model

for

vehicle

to

vehicle

communication

was

first

proposed

by

Akki

and

Haber

and

its

statistical

properties

reported

in

This

model

covers

Rayleigh

distribution

only

with

both

inphase

and

quadrature

components

having

identical

variances.

Matolak

et

al

after

performing

measurements

in

five

different

cities,

models

the

channel

as

Weibull

fading

channel.

Based

on

the

work

of

many

simulators

have

been

designed

and

implemented.

Cox

et

al

presented

a

discrete

line

spectrum

based

approach

to

simulate

the

channel.

The

work

in

was

based

on

sum

of

sinusoids

(SOS)

approach

for

simulator

design.

The

simulator

proposed

in

is

based

on

Kullback-Leibler

divergence

which

is

compared

with

IFFT

based

approach

of

simulator

design.

Borries

et

al

used

Gaussian

quadrature

rules

for

simulator

design.

Zaji

et

al

proposed

an

efficient

sum

of

sinusoids

(SOS)

based

approach

for

simulator

design.

All

the

simulator

design

approaches

mentioned

above

are

restricted

to

Rayleigh

fading

channel

only..

The

simulators

mentioned

above

may

be

classified

into

three

categories.

The

first

category

uses

the

sum

of

sinusoid

(SoS)

approach

for

generating

the

fading

channel

coefficients.

This

approach

has

the

drawback

of

multiple

Sine

function

calls

which

makes

it

computationally

expensive

to

implement

in

real

time.

Secondly

this

kind

of

simulator

does

not

produce

the

channel

with

the

statistical

properties

that

match

with

the

theoretical

values.

Another

class

of

simulators

uses

IFFT

based

approach

to

generate

the

required

channel

coefficients.

This

approach

is

computationally

efficient.

The

drawback

is

that

it

works

on

a

block

of

data

and

can

not

be

used

for

the

streaming

data

in

real

time.

The

approach

used

in

this

paper,

is

the

Filter

based

approach.

This

is

computationally

efficient

as

well

as

produces

channel

coefficients

having

more

accurate

statistical

properties..

The

proposed

simulator

is

a

modified

form

of

the

simulator

described

in

The

proposed

simulator

uses

a

generalized

Wideband

Nakagami

Hoyt

with

diffused

line

of

sight

channel

model

for

Vehicle

to

Vehicle

communication

environment

to

generate

real

time

fading

data.

It

covers

Rayleigh,

Rice,

Rice-Hoyt,

Lognormal

and

static

channels

as

its

special

cases.

The

multipaths

have

been

modeled

as

Tap

Delay

Line

(TDL)

filter.

Efficient

implementation

of

optimal

TDL

filter

has

been

performed

over

the

DSP

processor.

The

novel

simulator

implementation

uses

two

DSP

and

boards

along

with

one

wide

bandwidth

(Microline

I/O

daughter

board.

To

the

best

of

authors'

knowledge,

to

date

such

real

time

simulator

has

not

been

proposed

and

implemented..

The

remainder

of

this

paper

is

organized

as

follows.

Section

presents

the

brief

overview

of

the

proposed

Diffused

Nakagami-Hoyt

channel

model.

Section

describes

the

channel

simulator

design

philosophy

and

architecture.

Section

shows

the

outcome

of

the

simulator

and

their

comparison

with

the

analytical

expressions.

Finally,

Section

concludes

the

paper.

In

this

paper,

design

and

implementation

of

an

efficient

real

time

wideband

simulator

has

been

discussed.

The

simulator

was

run

in

real

time

with

a

known

input

and

the

output

data

was

analyzed.

The

TDL

filter

has

been

optimally

implemented

over

DSP.

The

output

of

the

filter

has

been

verified

by

comparing

the

simulator

output

with

MATLAB.

The

pipelined

architecture

of

the

processor

and

the

circular

buffer

have

been

efficiently

utilized.

The

channel

coefficients

have

been

generated

and

analyzed.

The

BPSK

modulated

data

has

been

input

and

the

output

has

been

stored.

The

bit

error

rate

has

been

measured

and

compared

with

the

theoretical

data

to

verify

the

validity

of

the

channel

simulator.

O O O O O O O O B O O O O O O O O O O O O O O O B O B O O O B I O O O B I O O B O O O O B O O O O O O O O O O B I O O O O O O O O O O B O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B


Evolving A-Type Artificial Neural Networks Abstract We investigate Turing's notion of an A-type artificial neural network. We study a refinement of Turing's original idea, motivated by work of Teuscher, Bull, Preen and Copeland. Our A-types can process binary data by accepting and outputting sequences of binary vectors; hence we can associate a function to an A-type, and we say the A-type represents the function. There are two modes of data processing: clamped and sequential. We describe an evolutionary algorithm, involving graph-theoretic manipulations of A-types, which searches for A-types representing a given function. The algorithm uses both mutation and crossover operators. We implemented the algorithm and applied it to three benchmark tasks. We found that the algorithm performed much better than a random search. For two out of the three tasks, the algorithm with crossover performed better than a mutation-only version. In this paper we report on our investigations into one of Alan Turing's contributions to artificial intelligence. In 1948 Turing introduced a type of artificial neural network (ANN), which he called an A-type unorganised machine. Motivated by his work and by work of Teuscher, Bull, Preen and Copeland (see Section 2), we study a refinement of Turing's notion, which we call an A-type.. A-types can be used to process binary data: with suitable conventions involving input and output nodes, one can input a string of binary vectors into an A-type and receive a string of binary vectors as output. Hence we can associate a function to an A-type; we say that the A-type represents this function. We devised an evolutionary algorithm (EA) to design an A-type that represents a given function f . We use a graph-based representation for A-types, and our EA-and in particular, our crossover operator-is based on graph-theoretic ideas. We implemented our algorithm and applied it to three benchmark problems.. Turing's research on A-types is of great historical interest. As the centenary of his birth approaches, it is fitting to apply modern ideas-such as the theory of non-linear dynamical systems-to his ground-breaking work. A-types are an excellent test-bed for these ideas: they are composed of neurons with a very simple firing rule and they are easy to program, but they are also powerful. In this paper we adapt some existing ideas such as graph-based chromosomes and sequential input to the setting of A-types. The use of sequential input mode here brings up some new problems which motivated us to introduce a new kind of neuron, delay nodes, not originally envisaged by Turing (see Sec-tion 3.5). Our graph-based EA works in the settings of both sequential and clamped input.. In Section 2 we give a brief survey of previous work on A-types. In Section 3 we present our interpretation of A-types, and we describe our EA in Section 4. Section 5 contains the results of our experimental work.. Our investigations are mainly at the proof-of-concept level. Our EA has many parameters and we chose their values in an ad-hoc fashion to ensure that solutions were quickly discovered reasonably often; we did not search systematically for the optimum values (but see Sections 5.6 and 6.1). We devised a graph-based EA for finding A-types that represent a given function. When applied to the three benchmark problems, the EA performed considerably better than a purely random search. For clamped nidentity and n-carry, the full version of the EA performed better than the mutation-only version. Our algorithm worked in both the clamped and the sequential settings. Size of solution n genetic_search_one mutation_search_one Fig. 26 The sizes of A-types found to represent n-carry using mutation search one and genetic search one. Again, the error bars show the 90% confidence interval using Student's t-test.. We now suggest directions for future research. Atypes are relatively simple, yet they are recurrent Boolean ANNs capable of representing any Boolean function and operating in a sequential mode. Consequently, we believe A-types are a useful tool for investigating the learning and behaviour of Boolean ANNs. In particular, the simplicity of A-types means that manipulations of their graphs are often straightforward to implement. We suggest two areas of future research with A-types: further investigations into evolutionary techniques, and using the symmetry of a concept function to improve the search for an A-type that represents that function.

O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


A Comprehensive Statistical Model for Cell Signaling Abstract-Protein signaling networks play a central role in transcriptional regulation and the etiology of many diseases. Statistical methods, particularly Bayesian networks, have been widely used to model cell signaling, mostly for model organisms and with focus on uncovering connectivity rather than inferring aberrations. Extensions to mammalian systems have not yielded compelling results, due likely to greatly increased complexity and limited proteomic measurements in vivo. In this study, we propose a comprehensive statistical model that is anchored to a predefined core topology, has a limited complexity due to parameter sharing and uses micorarray data of mRNA transcripts as the only observable components of signaling. Specifically, we account for cell heterogeneity and a multilevel process, representing signaling as a Bayesian network at the cell level, modeling measurements as ensemble averages at the tissue level, and incorporating patient-to-patient differences at the population level. Motivated by the goal of identifying individual protein abnormalities as potential therapeutical targets, we applied our method to the RAS-RAF network using a breast cancer study with 118 patients. We demonstrated rigorous statistical inference, established reproducibility through simulations and the ability to recover receptor status from available microarray data. C ELLS are complex molecular machines contained within phospholipid membranes that isolate a unique chemical environment. A key component of the cellular machinery is the set of protein signaling networks, which permit a cell to sense the internal and external environments and respond by altering metabolism and gene expression. Signaling networks comprise interacting signaling pathways, with each pathway containing a number of individual signaling proteins.. Signaling proteins can modify their behavior based on conformational changes induced by other signaling proteins. In the typical case, a kinase (a protein capable of adding a phosphate group to a protein) modifies its target protein by adding phosphate groups at serine or threonine amino acid residues. The modified protein undergoes a conformational change, activating its own kinase activity, leading to modification of a new target protein. This chain of phosphorylation causes a signal to be transduced through the cytosol of the cell, resulting in changes in enzymatic activity or activation or suppression of a transcriptional regulator (i.e., a transcription factor or cofactor). In addition to kinases, there are phosphatases that remove phosphate groups, thus reversing the signal from a kinase. Also, for certain signaling proteins, activity is generated by cleavage of a parent protein or dimerization, which is especially common for receptor tyrosine kinases that reside on the cell membrane and respond to external environmental cues, such as hormones or growth factors. Cell signaling processes play a central role in the etiology of many diseases, and signaling proteins provide a logical target for therapeutic intervention with numerous therapeutics under development [24] . The success of imatinib mesylate (IM, Gleevec) in treating chronic myelogenous leukemia has greatly increased the hope for targeted therapy, however, these new targeted therapeutics are designed to disrupt a single signaling protein. As studies in glioblastoma multiforme have demonstrated, each individual tumor has a different specific set of aberrant signaling proteins [23] , [26] , making it essential to identify in each individual which proteins need to be targeted for treatment.. The logical method to identify an aberrant signaling protein is to look for changes in protein posttranslational modifications (PTMs), since most signal propagation takes the form of phosphorylation changes in proteins or cleavage events changing protein localization and structure. However, these measurements are presently very limited in vivo. An alternative approach is to use the mature microarray technology targeted at mRNA transcripts, since transcriptional changes resulting from activation or suppression of transcriptional regulators are primary end points for many signaling processes. Microarray data coupled to models of signaling networks provide a potential avenue for identification of individual signaling protein abnormalities.. We have designed a statistical model for cell signaling, which accounts for cell heterogeneity, can be robustly learned from available microarray data and supports rigorous statistical inference. Our effort was mainly invested in laying out a comprehensive framework to identify and quantify aberrations in signaling. Consequently, we used prior knowledge and considered a documented core topology (Fig. 1 ) that is particular to our breast cancer study and available expression data. We followed a multilevel approach to elaborate the overall generative process starting from hidden phenotypes to final log-expressions with different statistical interpretations in cell, tissue, population, and measurement levels.. It is worth noting here that, for computational purposes, we constructed our Bayesian network formulation at the cell level, with parameter-free, linear, and generic transition probabilities as given in (2), which, it may be argued, oversimplify the underlying chemical processes. However, the overall model allows the user to incorporate his/her expert knowledge and to explain signaling dynamics with more complex, nonlinear choices, which in turn may enhance the predictive accuracy of the method. In fact, without sacrificing efficiency, one can assume alternative formulations to (2) that are still linear but favor the known dominance of one or more competing parents at crossing pathways; or, for instance, one can differentiate interactions at the signaling level from those at the level of transcription. To further enrich the model, one can even introduce extra parameters that can be validated as more protein data becomes available.. Note also that, due to lack of measurements on all hidden variables but ER, the biological validation of our model remains currently very limited. On the other hand, ER ground truth usually correlates with the majority of the genes. Thus, if the task were to predict ER statuses only, one would argue for simpler Bayesian approaches with performances comparable to ours. However, this argument should not lessen the utility of our model which lays out a comprehensive framework to infer on each hidden component. In that regard, our ER predictions we report here, should not be interpreted as an achievement, which otherwise could not be made, but rather a consistency check.. Finally, we have demonstrated model's identifiability, reproducibility through simulations and robustness under biologically meaningful revisions of topology. Using real patient data, we validated its ability to recover receptor status in a breast cancer study. As signaling plays a central role in the etiology of many diseases, identification of the aberrant proteins driving signaling errors will provide information for personalized therapeutic intervention. It is expected that this will improve patient prognosis and reduce undesirable side effects during treatment. Bayesian network models have been used in a wide variety of ways. For example, the relationships between nodes do not need to represent actual physical connections; consequently, Bayesian networks can model the effects of clinical variables on outcome, even relying on molecular data as well [18] . This can be viewed as a phenomenological perspective, where we abstract away the direct molecular causative agents, but retain predictive relationships between measured variables [5] , [27] . Bayesian networks have also been used to model traditional genetic networks, such as with time series data, where the upregulation of a gene is identified as a causative agent for expression of other genes [6] . Other approaches to creating robust models might be attempted. Ordinary differential equation (ODE) models, such as modeling of ERBB signaling response [1] , can capture great complexity, but they rely on large numbers of poorly determined parameters. This can limit their provability, since large ranges of parameter values on many components must be explored to guarantee uniqueness. Alternatively, networks can be reconstructed from limited measurements of protein state and abundance, such as from flow cytometry [25] , or from prior data on beliefs of connectivity [20] . In these cases, the goal is to construct the connectivity and flow of the network for a small number of proteins from proteomic measurements.. For inference on larger cell signaling networks, a number of alternative methods have been used. Matrix factorization has been used to determine activity on components of networks or in biological processes from microarray data, such as through Network Component Analysis [16] and other methods reviewed in [13] . Bayesian methods have been applied to estimate gene regulatory networks from microarray data [21] , from microarray data coupled to other information such as protein-protein interactions [9] , and from proteomic measurements of signaling species in the networks [20] .. Our work has two substantial differences to this previous work. First, we introduce the ensemble of cell models concept that captures biological heterogeneity. Second, we build a Bayesian network that realizes the molecular interactions of the biological network as reflected in transcriptional changes that can be measured routinely and globally.

O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Formal verification of function blocks applied to IEC 61131-3 Many industrial control systems use programmable logic controllers (PLCs) since they provide a highly reliable, off-the-shelf hardware platform. On the programming side, function blocks (FBs) are reusable components provided by the PLC supplier that can be combined to implement the required system behaviour. A higher quality system may be realized if the FBs are pre-certified to be compliant with an international standard such as IEC 61131-3. We present an approach: 1) to create complete and unambiguous FB requirements using tabular expressions; and 2) to verify the consistency and correctness of FB implementations in the PVS proof environment. We apply our approach to the examples in the informative Appendix F of the IEC 61131-3 standard. We examined the entire library of FBs and their supplied implementations described in structured text (ST) and function block diagrams (FBDs). Our approach identified issues in the informative examples, including: a) ambiguous behavioural descriptions; b) missing assumptions; and c) inconsistent implementations. We also proposed solutions to these issues. Many industrial control systems have replaced traditional analog equipment by components that are based upon programmable logic controllers (PLCs) to address increasing market demands for high quality [1] . Function blocks (FBs) are basic design units that implement the behaviour of a PLC, where each FB is a reusable component for building new, more sophisticated components or systems. Standards such as DO-178C [2] (in the aviation domain) and IEEE 7-4.3.2 [3] (in the nuclear domain) list acceptance criteria for mission-or safety-critical systems that practitioners need to comply with. Two important criteria are: 1) the system requirements are precise and complete; and 2) the system implementation exhibits behaviour that conforms to these requirements. In one of its supplements, DO-178C advocates the use of formal methods to construct, develop, and reason about the mathematical models of system behaviours. To this end, we present methods that support the use of formal notations for specifying the required behaviour of FBs, and for verifying that each FB (including composed FBs) complies with its requirements.. Tabular expressions [4, 5] (a.k.a., function tables or tables) have proven to be both practical and effective in formally documenting system requirements in industry [6, 7] . PVS [8] is a general purpose theorem prover that provides an integrated environment with mechanized support for writing specifications using tabular expressions and (higher-order) predicates, and for (interactively) proving that implementations satisfy the tabular requirements using sequent-style deductions. In this paper we report on using tabular expressions to formalize the requirements of FBs and on using PVS to verify their correctness (with respect to tabular requirements).. As a case study, we attempted to verify the FBs 1 listed in "Informative" Annex F of the 2003 version of IEC 61131-3 [9] as well as the FBs described in the standard itself. IEC 61131-3 is an important standard with over 20 years of use on critical systems running on PLCs. We had two reasons for choosing IEC 61131-3 for our case study. First of all, this provided a number of FBs that represent useful behaviours in a number of application domains, so our methods could be applied to FBs that we knew were representative of industrial use. Secondly, although the FBs of Annex F are not technically part of the standard as indicated by the labelled "Informative", the entire document, including all annexes, has become the de facto standard for FBs. PLC vendors have based their libraries on the FBs from Annex F, as well as those described in the body of the standard. The standard itself does not make any claim as to the completeness and appropriateness of the behaviour of FBs. In addition, no one has published a "validated and verified" version of the FBs in the standard. Thus, companies that develop mission-critical or safety-critical systems using PLCs had to qualify the behaviour of their libraries based on IEC 61131-3 (including Annex F), at considerable cost. If practitioners can use pre-defined and pre-verified FBs, then this will help raise the quality of FB-based implementations in industry without the overhead that would be required if each practitioner had to perform the verification separately.. Currently, some of the design specifications in the standard (expressed in source code) are incorrect, in that they are not what is commonly expected in practice. We believe that formal requirements of the FB behaviour, such as those provided by tabular expressions, help tool vendors and users of FBs have the same interpretations of the expected system behaviours. Also, formal descriptions are amenable to mechanized support such as PVS to verify the conformance of candidate implementations to a high-level, input-output requirements. For the purpose of this paper, we focus on FBs that are described in the more commonly used languages of structured texts (STs) and function block diagrams (FBDs). Note that two versions of IEC 61131-3 are cited here. The earlier version [9] has been in use since 2003. Most of the work reported in this paper relates to this version.. As we will see, a number of issues were uncovered in the FBs in the standard and in its informative Annex F. Our intent in the proceeding discussion is to illustrate how our methodology raised questions about some of these FBs. It is not a direct criticism of the standard, since the original mandate of the standard did not include the presentation of a pre-validated and pre-verified FB library. In fact, the standard does not attempt to define the required behaviour of each FB at the semantic level that we would expect from a requirements specification. Instead it uses code. These source programs are operational descriptions, making it hard to identify unexpected behaviour, and they are thus at an inappropriate level of abstraction for specifying requirements. Consequently, we had to provide the high-level requirements specifications based on our experience and on what we deduced was the intended behaviour of the FB. Readers may not agree with our version of the required behaviour, but we did make an honest attempt to define the behaviour that would be consistent with industrial norms. In any case, our motivation here is to demonstrate our methods, not to criticize the standard. We hope that readers will be interested that the methodology highlighted potential problems with FBs that have been in use for many years, and that this type of methodology can help us improve the quality of FB-based designs. In 2013, a new version of the standard was issued [10] , and this version did not include Annex F. Some of the FBs in the new version do still exhibit behaviour that we believe could be improved through use of this methodology. Many industrial control systems, especially safety-critical systems, require a high degree of confidence in the safety of the system. For those systems that are based upon PLCs, IEC 61131-3 provides several standard programming languages to describe the behaviour of the implemented function blocks (FBs). Since FBs have been growing in popularity, we present an approach to formalizing and verifying FBs using tabular expressions in the Prototype Verification System (PVS) environment. For STs, we provide a list of translation patterns. For the purpose of our verification, our rules of ST-to-PVS translation are designed to handle only the syntactic constructs of the ST language that are exploited in Annex F. That is, constructs that are supported by the ST language but not used in Annex F of the standard [9] (e.g., CASE statement, WHILE and REPEAT loops, etc.) are not currently handled by our translation rules. As mentioned earlier, our translation is still very useful since the Annex F example function blocks are commonly used in industry. As future work, we intend to extend our translation rules to cover the remaining programming constructs of the ST language, so as to be able to verify FBs using those constructs. For FBDs, we formalize each basic FB as a predicate, allowing for deriving the semantics of composite FBs via logical compositions. More importantly, for demonstrating how we can argue the correctness of the ST and FBD implementations, we provide a black-box input-output requirement specification for each FB in the form of tabular expressions. Finally, as a demonstration of the applicability and value of our methodology, we formally verified the consistency and correctness of the whole FB library in IEC 61131-3 2003 and its informative annex in PVS.. Using our approach, we identified several kinds of potential issues in the FBs in the IEC 61131-3 standard and in the informative annex (Annex F) of the 2003 version: ambiguous block behaviour, missing assumptions, initialization failure, division by zero, mismatched variable types, and erroneous implementations. We provided our suggested solutions for each of these issues to demonstrate that the methodology can help us identify inconsistencies, ambiguities, missing information and even technical errors, that may be difficult or too time consuming to find through manual analysis or by simply examining the FBs in order to understand their behaviour. As indicated, the primary purpose of this work was to demonstrate that this type of formal approach can work on real industrial examples, and that the results are both useful and based on sound principles. As a side benefit, if methodologies like this move into accepted practice, and applied to published libraries of functions and FBs, the FBs in international standards such as IEC 61131-3 could be reused safely in PLC-based control system development, and hardware manufacturers' implementations of the FBs could be tested against the black-box requirement specifications of the FBs. This would strengthen the benefits of being in compliance with such a standard, for developers and manufacturers alike.. In future work, we will extend our list of ST-to-PVS translation patterns to cover more syntactic features. We will also adapt, and possibly extend, the approach for verifying FBs described in another FB related standard, IEC 61499 that fits well with distributed control systems. There are many articles on formalizing and verifying PLC programs specified by programming languages covered in IEC 61131-3, such as sequential function charts (SFCs). Some approaches do this using model checking: e.g., to formalize a subset of the language of instruction lists (ILs) using timed automata, and to verify real-time properties in Uppaal [25] ; to automatically transform SFC programs into the synchronous data flow language of Lustre, amenable to mechanized support for checking properties [26] ; to translate ST and FBD into a synchronized data-flow language SIGNAL to compile and analyze the verification of specifications [27] ; to transform FBD specifications to Uppaal formal models to verify safety applications in the industrial automation domain [28] ; to provide the formal operational semantics of ILs which is encoded into the symbolic model checker Cadence SMV, and to verify rich behavioural properties written in linear temporal logic (LTL) [29] ; and to provide the formal verification of a safety procedure in a nuclear power plant (NPP) in which a verified Coloured Petri Net (CPN) model is derived by reinterpretation from the FBD description [30] ; to translate the algorithms of ladder diagrams (LDs) and timed FBs into finite state automata in which some properties are verified in SMV [31] ; There is also an integration of SMV and Uppaal to handle, respectively, untimed and timed SFC programs [32] .. Some other approaches adopt the verification environment of a theorem prover: e.g., to check the correctness of SFC programs, automatically generated from a graphical front-end, in Coq [33] ; and to formalize PLC programs using higher-order logic and to discharge safety properties in HOL [34] . These works are similar to ours in that PLC programs are formalized and supported for mechanized verifications of implementations. An algebra approach for PLC programs verification is presented in [35] . In [36] , a trace function method (TFM) based approach is presented to solve the same problem as ours.. Our work is inspired by [37] in that the overall system behaviour is defined by taking the conjunction of those of internal components (circuits in [37] or FBs in our case). Our proposed solutions to the timing issues of the PULSE timer are consistent with [38] . However, our approach is novel in that 1) we also obtain tabular requirements to be checked against, instead of writing properties directly for the chosen theorem prover or model checker; and 2) our formalization makes it easier to comprehend and to reason about properties of disjointness and completeness.. The related work is motivated by the lack of formal semantics for the programming notations defined in the standard, and attempts to remove ambiguities. However, the issues we found are not reported in the related work. We situate our work from three aspects: 1) extent of case study; 2) value of results; and 3) practical implication.. Extent of case study. Our approach is able to handle all ST and FBD programs that are listed in [9] , including its Annex F, whereas other work (e.g., [25, 27, 29, 31] ) focuses on limited language constructs or example FBs.. Value of results. To our knowledge, there is only a limited number of papers that illustrate the proposed verification approach via a case study, but none of them conduct a case study to the same extent as ours, let alone categorize the uncovered issues. In this paper, our results are based on the formalization and proofs of all FBs listed in the standard and its Annex F, whereas others (e.g., [32, 33] ) validate their approach via only a limited number of example blocks.

B I O B I O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O O O B I O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Domain adaptation of statistical machine translation with domain-focused web crawling Abstract In this paper, we tackle the problem of domain adaptation of statistical machine translation (SMT) by exploiting domain-specific data acquired by domainfocused crawling of text from the World Wide Web. We design and empirically evaluate a procedure for automatic acquisition of monolingual and parallel text and their exploitation for system training, tuning, and testing in a phrase-based SMT framework. We present a strategy for using such resources depending on their availability and quantity supported by results of a large-scale evaluation carried out for the domains of environment and labour legislation, two language pairs (EnglishFrench and English-Greek) and in both directions: into and from English. In general, machine translation systems trained and tuned on a general domain perform poorly on specific domains and we show that such systems can be adapted Lang Resources & Evaluation (2015) 49:147-193 DOI 10.1007 successfully by retuning model parameters using small amounts of parallel indomain data, and may be further improved by using additional monolingual and parallel training data for adaptation of language and translation models. The average observed improvement in BLEU achieved is substantial at 15.30 points absolute. Recent advances in statistical machine translation (SMT) have improved machine translation (MT) quality to such an extent that it can be successfully used in industrial processes (e.g., Flournoy and Duran 2009 ). However, this mostly happens only in specific domains where ample training data is available (e.g., Wu et al. 2008) . Using in-domain data for training has a substantial effect on the final translation quality: the performance of an SMT system usually drops when it is applied to data of a different nature than that on what it was trained (e.g., Banerjee et al. 2010) .. SMT is an instance of a machine learning application which in general works best if the data for training and testing are drawn from the same distribution (i.e., domain, genre, and style). In practice, however, it is often difficult to obtain sufficient amounts of in-domain data (in particular, parallel data required for translation and reordering models) to train a system with good performance for a specific domain. The main problem is usually vocabulary coverage: domain-specific texts typically contain a substantial amount of special vocabulary, which is not likely to be found in texts from other domains ). Additional problems can be caused by divergence in style or genre, where the difference is not only in lexis but also in other linguistic aspects such as grammar.. In order to achieve optimal performance, an SMT system should be trained on data from the same domain, genre, and style as it is intended to be applied to. For many domains, though, in-domain data of a sufficient size to train an SMT system with good performance is difficult to find. Recent experiments have shown that even small amounts of such data can be used to adapt an existing (general-domain) system to the particular domain of interest ). Sometimes, appropriate sources of such data come in the form of existing in-house databases and translation memories (He et al. 2010 ). An alternative option pursued in this paper is to exploit the constantly growing amount of publicly available text on the web, although acquiring data of a sufficient quality and quantity from this resource is a complicated process involving several critical steps (crawling, language identification, cleaning, etc.) .. In this research, we first present a strategy and relevant workflows for automatic webcrawling and cleaning of domain-specific data with limited manual intervention. These workflows are based on open-source tools and have also been deployed as web services in the context of the Panacea 1 research project (Poch et al. 2012) . One advantage of making the tools available as services is that chaining them together enables the building of dynamic and flexible workflows, which can always be improved by integrating new services and/or old legacy systems that may run on different technological platforms. Moreover, the user does not have to deal with technical issues regarding the tools, such as their installation, configuration, or maintenance.. These workflows are then employed to acquire monolingual and parallel data for two domains: environment (env) and labour legislation (lab), and two language pairs: English-French (EN-FR) and English-Greek (EN-EL). The crawled data is further exploited for domain adaptation of a general-domain SMT system in several ways: by domain-specific parameter tuning of the main log-linear model and by adaptation of its components. The evaluation experiments carried out in a total of eight evaluation scenarios (two domains, two language pairs, and both translation directions: to and from English) confirm substantial and consistent improvements in translation quality for all approaches compared to the baseline.. We explain the improvements brought about by analysing the experimental results in detail. In a nutshell, tuning for matching-domain training and test data results in weight vectors that trust (often long) translation table entries. Tuning with and for specific domains (while using generic training data) allows the MT system to stitch together translations from smaller fragments which, in this case, leads to improved translation quality. Such tuning requires only small development sets which can be harvested automatically from the web with minimal human intervention; no manual cleaning of the development data is necessary.. In addition, additional improvements are realised by using monolingual and/or parallel in-domain training data. Adaptation of language models focuses on improving translation fluency and lexical selection for the particular domain. Adaptation of the translation model then aims at reduction of the out-of-vocabulary (OOV) rate and adding domain-relevant translation variants. All the data sets are available via the European Language Resources Association (ELRA). This paper is an extended and updated version of our previous work published as Pecina et al. (2011 Pecina et al. ( , 2012a . Compared to these conference papers, we provide more details of the experiments, full results and a more thorough analysis and description of our findings. Some experiments are new, and not contained in the standalone papers. These include a comparison of various methods for adaptation of language models and translation models (including the state-of-the-art linear interpolation), as well as the comparison of OOV rate (i.e., the ratio of source words unknown to the translation model), language model perplexity measures, and average phrase length in the test set translations (cf . Table 15 ). Compared to the previous papers, the translation quality evaluation in this work is conducted on tokenized and lowercased translations to avoid any bias caused by recasing and detokenization. We also provide much longer descriptions of both related work as well as our data acquisition procedure. Finally, we formulate this paper as one concise yet coherent account of the full range of experiments carried out.. The remainder of the paper is organised as follows. After the overview of related work and description of the state-of-the-art in Sect. 2, we present our web-crawling procedure for monolingual and parallel data in Sect. 3, and the baseline system including its evaluation in Sect. 4. Section 5 is devoted to system adaptation by parameter tuning and Sect. 6 to adaptation of language and translation models. Section 7, which concludes the work, is followed by an Appendix containing formal definitions of the two domains relevant to our work and complete results of the main experiments.. 2 State-of-the-art and related work. In this section, we review the current state-of-the-art in the area of web crawling for monolingual as well as parallel data and briefly describe the main concepts of phrase-based SMT (PB-SMT) and its adaptation to specific domains. In the first main part of the paper (Sect. 4), we focused on a detailed exposition of the pipeline for acquisition of monolingual and parallel data from the World Wide Web. In the second part (Sects. 5-6), we added a thorough investigation of the impact of resources that can be generated using the pipeline, focusing in particular on major established/tried-and-tested approaches to domain adaptation of MT. We Table 14 (env domain only) based on general-domain data for training and tuning (B0), domain-specific parallel data for tuning (P1), additional monolingual data for language models (L2), and additional parallel data for the translation model (C2) Domain adaptation of statistical machine translation 183 discussed the effect of tuning and adaptation on SMT system weights, analysed the learning curves of parameter tuning, OOV rates, perplexity of the test data, and phrase length in translations produced during various stages of adaptation. The pipeline for the acquisition of domain-specific monolingual and parallel texts from the web is based on existing open-source tools for web crawling, text normalisation and cleaning, language identification, duplicate removal, and parallel sentence extraction. It is implemented as easy-to-use web services ready to be employed in industrial scenarios. It requires only limited human intervention for constructing the domain definition and the list of seed URLs, which can easily be tweaked and tuned to acquire texts with high accuracy of 94 %. This pipeline was applied to acquire domain-specific resources for adaptation of a general-domain SMT system. We crawled monolingual and parallel data for two language pairs (English-French, English-Greek) and two domains (environment, labour legislation), which allowed us to perform a large-scale evaluation using a total of eight test scenarios. The acquired data sets are available from ELRA.. Our domain-adaptation experiments focused on the following three components of a PB-SMT model: parameters of the log-linear combination and their optimisation, language model, and translation model. First, we confirmed the observation from previous research that systems trained and tuned on general domain perform poorly on specific domains. This finding is not surprising, but the amount of loss and the fact that it is observed consistently was rather unexpected. The average absolute decrease in BLEU in all the domain-specific evaluation scenarios was 21.82 points (37.86 % relative).. We confirm the results of previous research on tuning-only adaptation. Tuning the general domain-trained systems on specific target domain data recovers a significant amount of the loss. Several hundreds of sentence pairs used as development data improved the BLEU score of the baseline tuned on generaldomain data by 6.64 points absolute (24.82 % relative) on average. A detailed analysis of the model parameters and phrase length distribution in translations of the test data found that a system trained and tuned on general domain data strongly prefers long and few phrases in the output translations, and thus underperforms on specific domains where such phrases do not occur so frequently. In contrast, the same systems tuned on domain-specific data produce output translations from shorter phrases, allow specific reordering and perform significantly and consistently better on specific domains. Importantly, our findings show that the development data does not have to be manually cleaned and corrected, as parameter tuning on the development set (here, using MERT) is quite tolerant to imperfect translations and eventual noise in the development sets. Cross-domain tuning on a different set also offers a good solution when no in-domain development data is available, especially when the domains differ in a similar way. This step has the effect of tweaking the original generaldomain system towards shorter phrases and it does not matter much which different development sets are used.. The experiments with language model adaptation confirmed previous results. Linear interpolation of the general-domain and domain-specific models increased translation quality by a further 4.88 BLEU points absolute (14.95 % relative) compared to the general-domain systems tuned on in-domain development sets on average and significantly outperformed other techniques (concatenation of training data and log-linear combination of the two models). Adaptation of translation models (using 7,000-20,000 acquired sentence pairs) increased BLEU scores by 4.56 points absolute (13.87 % relative) compared to the general-domain systems tuned on in-domain development sets. In this case, linear interpolation and loglinear combination produced similar results. In the combined approach, we observed that the effect of using in-domain monolingual and parallel data is largely independent and does not cancel out when these two types of resources are used at the same time. The final BLEU scores increased by 3.78 points absolute (9.66 % relative) with respect to the language-model-adapted systems, by 8.66 points absolute (26.43 % relative) with respect to the in-domain tuned systems, and by 15.30 points absolute (58.37 % relative) with respect to the general-domain baseline, all on average.. The pipeline for domain-focused web crawling described in this work proved to be very successful in acquisition of domain specific data-both monolingual and parallel. The experiments then showed a high impact of the acquired resources on domain adaptation of MT. We mainly concentrated on parameter tuning and analysis of its effects. Although we did not especially focuse on adaptation of language models and translation models, the acquired data also significantly improved these components and translation quality in general. ecosystems and biodiversity, which mainly refer to pollution control and remediation, legislation well as to resource conservation and management. Texts on natural disasters and their effects on social life are also relevant.

B I O B I I O O B I O O O O O O O O O B I O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Importance Sampling of Word Patterns in DNA and Protein Sequences Monte Carlo methods can provide accurate p-value estimates of word counting test statistics and are easy to implement. They are especially attractive when an asymptotic theory is absent or when either the search sequence or the word pattern is too short for the application of asymptotic formulae. Naive direct Monte Carlo is undesirable for the estimation of small probabilities because the associated rare events of interest are seldom generated. We propose instead efficient importance sampling algorithms that use controlled insertion of the desired word patterns on randomly generated sequences. The implementation is illustrated on word patterns of biological interest: Palindromes and inverted repeats, patterns arising from position specific weight matrices and co-occurrences of pairs of motifs. Searching for matches to a word pattern, also called a motif, is an important task in computational biology. The word pattern represents a functional site, such as a transcription factor binding site (TFBS) in a promoter region of a DNA sequence or a ligand docking site in a protein sequence. Statistical significance of over-representation of these word patterns provides valuable clues to biologists and as a result, there have been a lot of work done on the use of asymptotic limiting distributions to approximate these p-values, see Prum et al. (1995) , Reinert et al. (2000) , Régnier (2000) , Robin et al. (2002) , Huang et al. (2004) , Leung et al. (2005) , Mitrophanov and Borodovsky (2006) , Pape et al. (2008) and references therein.. However, the approximations may not be accurate for short words or for words consisting of repeats and most theoretical approximations work only in specific settings. String-based recursive methods can provide exact p-values, see for example Gusfield (1997) , but they can be computationally expensive when the number of words in the word pattern is large.. Direct Monte Carlo algorithms for estimating p-values of word patterns are easy to implement but are inefficient for the estimation of very small p-values because in such cases, almost all the simulated sequences do not contain the required number of word patterns. We propose in this paper importance sampling algorithms that insert the desired word patterns either randomly or controlled by a hidden Markov model, on the simulated sequences. The algorithms are described in Section 2 and are illustrated on several word patterns of biological interest: Palindromes and inverted repeats in Section 3, high-scoring words with respect to position specific weight matrices in Section 4 and co-occurrences of motifs in Section 5.. Numerical results show that variance reduction of several orders of magnitude are achieved when applying the proposed importance sampling algorithms on small p-values. The technical details are consolidated in the appendices and include a proof of the asymptotic optimality of the importance sampling algorithms, in Appendix D. The examples given here are not meant to be exhaustive but they do indicate how we can proceed in situations not covered here. For example, if we would like the order of the two words in a CRM to be arbitrary, we can include an additional permutation step in the construction of the word bank. In Section 5.2, we also showed how to simulate p-values of the maximum count over a set of word patterns. As we gain biological understanding, the models that we formulate for DNA and protein functional sites become more complex. Over the years, they have evolved from deterministic words to consensus sequences to PSWMs and then to motif modules. As probabilistic models for promoter architecture gets more complex and context specific, importance sampling methods are likely to be more widely adopted in the computation of p-values.

B I O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O