Verification

of

Fault-Tolerance

and

Real-Time

Zhiming

Liu

Department

of

Mathematics

and

Computer

Science

University

of

Leicester

Mathai

Joseph

Department

of

Computer

Science

University

of

Warwick

Abstract

A

transformational

method

is

given

for

specifying

and

verifying

fault-tolerant

real-time

programs

Such

a

program

needs

to

be

provably

correct

according

to

its

both

functional

and

real-time

requirements

despite

of

the

presence

of

possible

system

failures

The

paper

demonstrates

that

a

suitably

expressive

logic

for

real-time

systems

allows

to

naturally

model

the

state

changes

by

system

failures

and

determine

their

effect

on

the

functional

and

real-time

properties

of

executions

Introduction

Consider

the

development

of

a

concurrent

program

by

a

sequence

of

steps

of

refinement

from

a

requirement

specification

to

an

executable

implementation

Each

step

in

such

a

development

constructs

a

lower

level

specification

Pl

from

a

higher

level

specification

Ph

and

it

must

be

provable

that

Pl

refines

Ph

denoted

Pl

v

Ph.

The

refinement

relation

Pl

v

Ph

is

reflexive

-LRB-

P

v

P

for

any

specification

P

-RRB-

and

transitive

-LRB-

if

Pll

v

Pl

and

Pl

v

Ph

then

Pll

v

Ph

-RRB-

Different

frameworks

and

refinement

calculi

have

been

used

to

prove

that

one

specification

refines

another

-LSB-

-RSB-

The

refinement

steps

described

above

assume

that

the

implementation

Pl

is

executed

on

a

fault-free

system

i.e.

each

operation

in

Pl

is

executed

correctly

according

to

its

defined

semantics

However

concurrent

programs

may

be

run

on

systems

which

exhibits

various

failures

referred

to

here

as

physical

faults

as

processors

may

fail

channels

may

lose

messages

and

memory

may

be

corrupted

Thus

if

the

execution

of

Pl

suffers

from

the

effects

of

such

faults

its

behaviour

may

not

satisfy

Ph

even

though

Pl

v

Ph

can

be

proved

This

paper

presents

transformational

framework

which

assumes

that

the

physical

faults

of

a

system

are

modelled

as

being

caused

by

a

set

F

of

fault

operations

which

perform

state

transformations

in

the

same

way

as

the

ordinary

program

operations

The

effect

of

the

faults

F

on

the

execution

of

a

program

P

can

be

modelled

by

a

transformation

F

of

P

into

an

F

affected

version

F

-LRB-

P

F

-RRB-

In

an

atomic

action

system

model

the

F

affected

version

F

-LRB-

P

F

-RRB-

is

simply

the

union

of

the

operations

of

P

and

F

and

these

This

is

an

extended

version

of

a

paper

to

be

presented

at

the

Annual

International

Symposium

on

Fault-Tolerant

Computing

Sendai

Japan

in

June

This

report

has

also

appeared

as

of

the

Department

of

Computer

Science

University

of

Warwick

are

executed

in

an

interleaved

manner

Fault-tolerance

is

achieved

if

a

specification

Pl

is

an

F

tolerant

implementation

of

a

specification

-LRB-

program

-RRB-

Ph

and

the

F

affected

version

F

-LRB-

Pl

F

-RRB-

refines

Ph.

Fault-tolerant

systems

often

also

have

real-time

constraints

So

it

is

important

that

the

timing

properties

of

a

program

are

refined

along

with

the

fault-tolerant

and

functional

properties

defined

in

a

program

specification

This

paper

also

extends

the

transformational

approach

by

adding

time

bounds

to

actions

This

will

allow

the

fault-tolerant

redundant

actions

-LRB-

such

as

checkpointing

actions

and

recovery

actions

-RRB-

to

be

specified

with

time

constraints

The

method

presented

in

the

paper

is

based

on

the

following

observations

A

requirement

specification

Spec

for

a

program

P

is

generally

refined

into

a

canonical

specification

S

-LRB-

Ph

-RRB-

which

can

be

encoded

as

an

action

system

Ph.

Analysis

of

the

possible

failures

in

the

system

-LRB-

perhaps

in

consultation

with

system

engineers

-RRB-

provides

the

specification

F

of

the

fault-environment

Ph

is

then

refined

into

an

F

tolerant

implementation

Pl

For

real-time

fault-tolerant

programs

timing

assumptions

about

faults

as

well

as

functional

effects

of

faults

have

to

be

specified

so

that

to

achieve

and

verify

fault-tolerant

refinements

with

real-time

constraints

Redundant

operations

for

fault-tolerance

have

to

be

timingly

constrained

to

meet

the

original

deadlines

given

in

the

high

level

specification

In

general

the

redundant

operations

are

required

to

satisfy

the

condition

that

after

a

fault

occurs

the

system

recovers

from

an

error-free

and

consistent

state

within

a

time

bound

which

takes

account

of

the

possible

wasted

computation

time

caused

by

the

fault

and

recovery

It

is

clear

that

fault-tolerance

in

real-time

systems

can

only

be

achieved

based

on

reasonable

timing

assumptions

about

the

occurrences

of

faults

Therefore

we

should

be

able

to

specify

various

assumptions

of

this

kind

Furthermore

for

a

given

high

level

specification

either

more

powerful

-LRB-

or

faster

-RRB-

machines

are

needed

or

the

original

deadlines

in

the

high

level

specification

are

relatively

loose

to

allow

fault-tolerant

implementations

We

shall

use

the

example

of

the

processor-memory

interface

to

illustrate

these

points

Program

Specification

and

Refinement

This

section

introduces

a

computational

model

and

a

specification

language

that

will

be

used

in

this

paper

Action

system

models

A

program

will

be

represented

as

an

action

system

which

is

a

pair

P

-LRB-

I

O

-RRB-

consisting

of

an

initial

condition

I

and

a

set

O

of

atomic

operations

on

a

finite

set

v

of

state

variables

A

state

s

of

program

P

is

a

mapping

from

v

to

an

associated

set

of

values

-LRB-

i.e.

the

value

domain

-RRB-

D

We

use

s

-LSB-

x

-RSB-

for

x

v

to

denote

the

value

of

x

in

state

s

The

initial

condition

I

is

a

first

order

predicate

with

free

variables

in

v

It

defines

the

permitted

initial

states

from

which

execution

of

P

can

start

Each

atomic

operation

o

O

is

represented

by

a

guarded

nondeterministic

multiple

assignment

g

x

Q

where

x

is

a

vector

of

some

state

variables

in

v

is

a

vector

of

logical

variables

and

Q

is

a

first

order

predicate

over

state

variables

and

logical

variables

An

operation

o

is

enabled

in

a

state

s

if

the

guard

g

of

o

is

true

in

s

and

there

is

a

possible

next

state

t

such

that

ffl

the

variables

in

x

are

assigned

the

values

and

all

other

variables

remain

unchanged

t

-LSB-

y

-RSB-

-LRB-

-LRB-

i

-RRB-

if

y

is

the

ith

element

x

-LRB-

i

-RRB-

of

x

s

-LSB-

y

-RSB-

otherwise

ffl

the

predicate

Q

is

satisfied

by

t

When

an

operation

o

is

enabled

it

can

be

executed

and

will

change

the

current

state

into

a

possible

next

state

An

atomic

operation

may

introduce

infinite

nondeterminism

This

expressive

power

is

needed

to

represent

a

fault

by

an

atomic

operation

which

may

damage

the

program

state

by

setting

a

variable

to

any

value

except

the

correct

ones

A

deterministic

assignment

g

v

e

is

a

special

case

of

a

nondeterministic

assignment

Also

the

guard

g

of

an

operation

may

be

omitted

when

it

is

the

constant

true

Because

of

the

atomicity

of

operations

a

behaviour

of

program

P

can

be

represented

as

an

infinite

sequence

oe

of

states

where

is

the

initial

state

which

satisfies

the

initial

condition

I

and

each

si

i

is

obtained

from

si

by

either

a

stuttering

step

i.e.

si

si

or

by

executing

an

operation

o

of

program

P

The

set

of

all

the

behaviours

of

a

program

is

stuttering

closed

if

an

infinite

state

sequence

oe

is

a

behaviour

of

the

program

then

so

is

any

behaviour

obtained

from

oe

by

adding

or

deleting

a

finite

number

of

stuttering

steps

A

behaviour

is

terminating

if

it

has

an

infinite

sequence

of

stuttering

steps

Example

Consider

a

simple

processor-memory

interface

The

processor

issues

read

and

write

operations

that

are

executed

by

the

memory

Such

an

interface

consists

of

two

registers

represented

by

the

following

two

state

variables

op

Set

by

the

processor

to

indicate

the

desired

operation

and

reset

by

the

memory

after

executing

the

operation

Its

value

space

is

fready

read

writeg

val

Set

by

the

processor

to

indicate

the

value

to

be

written

by

a

write

and

set

by

the

memory

to

return

the

result

of

a

read

Assume

that

its

value

space

is

the

integers

Z.

To

describe

the

interface

as

an

action

system

we

introduce

an

-LRB-

internal

-RRB-

variable

mem

with

value

space

Z

to

denote

the

contents

of

the

memory

Let

I

-LRB-

op

ready

-RRB-

Rp

-LRB-

op

ready

-RRB-

op

read

W

p

-LRB-

op

ready

-RRB-

-LRB-

op

val

-RRB-

-LRB-

w

v

-RRB-

-LRB-

w

write

v

Z

-RRB-

Rm

-LRB-

op

read

-RRB-

-LRB-

op

val

-RRB-

-LRB-

ready

mem

-RRB-

Wm

-LRB-

op

write

-RRB-

-LRB-

op

mem

-RRB-

-LRB-

ready

val

-RRB-

O

fRp

W

p

Rm

Wmg

The

interface

can

be

described

as

the

program

InterFace

-LRB-

I

O

-RRB-

Reasoning

about

action

systems

in

TLA

We

shall

use

TLA

-LSB-

-RSB-

to

reason

formally

about

the

properties

of

an

action

system

Models

of

TLA

are

defined

as

infinite

sequences

of

states

oe

where

each

si

is

a

state

over

a

set

of

state

variables

A

property

of

program

P

is

represented

by

a

formula

which

defines

a

set

of

behaviours

of

P

An

action

formula

A

is

a

boolean-valued

expression

which

specifies

the

relation

between

the

values

of

variables

before

the

execution

of

an

operation

and

the

values

of

primed

variables

after

the

execution

of

the

operation

It

is

interpreted

over

pairs

of

states

hs

j

A

such

that

A

holds

when

the

variables

v

in

A

are

assigned

values

according

to

state

s

and

the

primed

variables

j

x

vg

in

A

are

assigned

values

according

to

state

hs

is

called

an

A-step

if

hs

j

A.

Formulas

in

TLA

are

constructed

from

action

formulas

in

the

following

way

An

action

formula

A

is

a

formula

A

is

satisfied

by

a

sequence

oe

of

program

P

denoted

oe

j

A

if

j

A

A

first

order

state

predicate

is

a

particular

action

formula

which

does

not

have

primed

variables

oe

satisfies

if

the

initial

state

of

oe

satisfies

i.e.

j

If

is

a

formula

so

is

-LRB-

always

-RRB-

oe

satisfies

if

all

the

suffixes

of

oe

satisfy

The

formula

-LRB-

eventually

-RRB-

is

defined

to

as

and

this

is

satisfied

by

oe

if

there

is

a

suffix

of

oe

which

satisfies

Formulas

can

be

composed

using

the

first

order

connectives

and

-RRB-

with

their

standard

semantics

Quantification

-LRB-

i.e.

-RRB-

is

possible

over

logical

variables

whose

values

are

fixed

over

states

and

over

state

variables

whose

values

can

change

from

state

to

state

To

specify

stuttering

steps

in

the

execution

of

a

program

we

define

the

following

notation

for

an

action

formula

A

and

a

vector

u

of

variables

-LSB-

A

-RSB-

u

A

unch

-LRB-

u

-RRB-

where

unch

-LRB-

u

-RRB-

-LRB-

x

-RRB-

Transforming

action

systems

to

TLA

formulas

The

relationship

between

action

systems

and

TLA

can

be

defined

in

a

relatively

straightforward

way

A

syntactic

operation

o

g

x

Q

in

an

action

system

P

corresponds

to

an

equivalence

class

of

action

formulas

For

each

action

formula

A

in

this

class

and

each

pair

hs

of

states

over

the

state

variables

v

of

P

execution

of

o

in

state

s

terminates

in

state

iff

hs

is

an

A-step

The

operation

o

can

be

translated

into

an

action

formula

-LRB-

o

-RRB-

which

is

a

representative

of

the

equivalence

class

of

action

formulas

corresponding

to

o

-LRB-

o

-RRB-

g

Q

-LSB-

-RSB-

unch

-LRB-

v

n

x

-RRB-

where

are

the

primed

counterparts

of

x

and

v

nx

is

the

complement

of

the

variables

x

over

the

program

variables

v

of

P

Action

o

is

enabled

only

when

g

holds

When

o

is

enabled

and

executed

the

program

state

is

changed

so

that

the

new

values

of

the

variables

x

are

related

to

the

old

values

in

the

way

defined

by

Q

-LSB-

-RSB-

substituting

each

-LRB-

i

-RRB-

by

x

-LRB-

i

-RRB-

and

the

values

of

any

variable

outside

x

remain

unchanged

In

the

examples

of

this

paper

we

will

simply

omit

the

unch

-LRB-

v

n

x

-RRB-

part

when

we

specify

operations

Given

an

action

system

-LRB-

program

-RRB-

P

-LRB-

I

O

-RRB-

let

NP

-LRB-

o

-RRB-

This

is

the

state-transitionrelation

for

the

atomic

operations

of

P

The

exact

specification

of

P

is

expressed

by

the

formula

-LRB-

P

-RRB-

I

-LSB-

NP

-RSB-

v

This

formula

specifies

all

the

possible

sequences

of

values

that

may

be

taken

by

the

state

variables

including

the

internalvariables

Using

existentialquantification

to

hide

the

internal

variables

x

-LRB-

xn

-RRB-

v

the

-LRB-

external

-RRB-

safety

specification

of

P

is

given

as

-LRB-

P

-RRB-

-LRB-

P

-RRB-

A

behaviour

satisfies

-LRB-

P

-RRB-

iff

there

is

a

sequence

of

values

that

can

be

assigned

to

xi

i

n

such

that

the

so

obtained

behaviour

makes

-LRB-

P

-RRB-

true

Example

-LRB-

continued

-RRB-

For

the

processor-memory

interface

let

-LRB-

Rp

-RRB-

-LRB-

op

ready

-RRB-

-LRB-

read

-RRB-

-LRB-

W

p

-RRB-

-LRB-

op

ready

-RRB-

-LRB-

write

-RRB-

-LRB-

Z

-RRB-

-LRB-

Rm

-RRB-

-LRB-

op

read

-RRB-

-LRB-

ready

-RRB-

-LRB-

mem

-RRB-

-LRB-

Wm

-RRB-

-LRB-

op

write

-RRB-

-LRB-

ready

-RRB-

-LRB-

val

-RRB-

Then

-LRB-

InterFace

-RRB-

-LRB-

op

ready

-RRB-

-LSB-

-LRB-

Rp

-RRB-

-LRB-

W

p

-RRB-

-LRB-

Rm

-RRB-

-LRB-

Wm

-RRB-

-RSB-

v

where

v

fop

val

memg

From

the

user

's

point

of

view

mem

in

this

specification

is

an

internal

variable

Thus

we

have

-LRB-

InterFace

-RRB-

-LRB-

op

ready

-RRB-

-LSB-

-LRB-

Rp

-RRB-

-LRB-

W

p

-RRB-

-LRB-

Rm

-RRB-

-LRB-

Wm

-RRB-

-RSB-

v

Note

that

in

the

syntax

of

action

systems

there

is

no

equivalent

to

such

internal

variables

It

would

be

possible

to

rewrite

the

specification

P

-LRB-

I

O

-RRB-

of

program

P

as

P

var

x

-LRB-

I

O

-RRB-

which

makes

explicit

the

declaration

of

internal

variables

However

we

shall

just

leave

the

treatment

of

internal

variable

declarations

informal

Imposing

fairness

properties

Formulas

-LRB-

P

-RRB-

and

-LRB-

P

-RRB-

are

safety

properties

i.e.

they

are

satisfied

by

an

infinite

behaviour

iff

they

are

satisfied

by

every

finite

initial

portion

of

the

behaviour

Safety

properties

allow

behaviours

in

which

a

system

performs

correctly

for

a

while

and

then

leaves

the

values

of

all

variables

unchanged

Such

behaviours

are

undesirable

in

distributed

systems

and

they

can

be

ruled

out

by

adding

fairness

properties

For

an

action

formula

A

let

the

state

predicate

Enabled

A

be

true

for

state

s

if

there

exists

a

state

t

such

that

-LRB-

s

t

-RRB-

is

an

A-step

For

a

vector

u

of

variables

in

v

let

-LSB-

A

-RSB-

u

where

hAiu

A

unch

-LRB-

u

-RRB-

Weak

fairness

WFu

-LRB-

A

-RRB-

-LRB-

-RRB-

-LRB-

EnabledhAiu

-RRB-

Strong

fairness

SFu

-LRB-

A

-RRB-

-LRB-

-LSB-

Aiu

-RRB-

-LRB-

EnabledhAiu

-RRB-

The

weak

fairness

condition

WFu

-LRB-

A

-RRB-

says

that

from

any

point

in

a

behaviour

either

hAiu

will

be

disabled

infinitely

often

or

an

hAiu-step

will

eventually

occur

The

strong

fairness

condition

WFu

-LRB-

A

-RRB-

says

that

from

any

point

in

a

behaviour

either

hAiu

is

enabled

only

finitely

often

or

a

hAiu-step

will

eventually

occur

The

safety

specifications

-LRB-

P

-RRB-

and

-LRB-

P

-RRB-

are

usually

strengthened

by

conjoining

them

with

one

or

more

fairness

properties

The

canonical

forms

of

the

exact

and

external

specifications

of

a

P

are

respectively

-LRB-

P

-RRB-

I

-LSB-

NP

-RSB-

v

L

and

S

-LRB-

P

-RRB-

-LRB-

I

-LSB-

NP

-RSB-

v

L

-RRB-

where

x

xng

is

the

tuple

of

the

internal

variables

of

P

the

action

formula

NP

defines

the

nextstate

relation

of

P

v

is

the

set

of

variables

of

P

and

L

is

a

conjunction

of

the

fairness

properties

of

action

formulas

which

defines

the

fairness

requirement

of

P

Example

-LRB-

continued

-RRB-

For

the

processor-memory

interface

Enabledh

-LRB-

Rm

-RRB-

iv

op

read

Enabledh

-LRB-

Wm

-RRB-

iv

op

write

WFv

-LRB-

-LRB-

Rm

-RRB-

-RRB-

-LRB-

-LRB-

Rm

-RRB-

i

-RRB-

-LRB-

-LRB-

op

read

-RRB-

-RRB-

SFv

-LRB-

-LRB-

Rm

-RRB-

-RRB-

-LRB-

-LRB-

Rm

-RRB-

i

-RRB-

-LRB-

-LRB-

op

read

-RRB-

-RRB-

Similarly

we

can

obtain

WFv

-LRB-

-LRB-

Wm

-RRB-

-RRB-

and

SFv

-LRB-

-LRB-

Wm

-RRB-

-RRB-

We

assume

weak

fairness

conditionsfor

-LRB-

Rm

-RRB-

and

-LRB-

Wm

-RRB-

L

WFv

-LRB-

-LRB-

Rm

-RRB-

-RRB-

WFv

-LRB-

-LRB-

Wm

-RRB-

-RRB-

Given

an

action

system

P

with

specification

S

-LRB-

P

-RRB-

let

a

property

of

P

be

described

by

the

formula

P

To

prove

that

P

satisfies

property

P

is

to

prove

in

TLA

the

validity

of

the

implication

S

-LRB-

P

-RRB-

-RRB-

P

For

example

a

liveness

-LRB-

progress

-RRB-

property

often

has

the

form

-LRB-

read

as

leads-to

-RRB-

and

this

can

be

defined

as

-LRB-

-RRB-

-RRB-

To

prove

that

P

satisfies

this

property

is

just

to

prove

S

-LRB-

P

-RRB-

-RRB-

-LRB-

-RRB-

Refinement

mapping

The

relation

v

characterises

refinement

i.e.

that

program

correctly

implements

Let

S

-LRB-

-RRB-

-LSB-

-RSB-

and

S

-LRB-

-RRB-

-LSB-

-RSB-

be

canonical

specifications

of

and

respectively

where

x

xng

y

ymg

Then

the

refinement

relation

is

formalised

as

vs

iff

S

-LRB-

-RRB-

-RRB-

S

-LRB-

-RRB-

The

refinement

relation

v

can

be

proved

as

the

implication

S

-LRB-

-RRB-

-RRB-

S

-LRB-

-RRB-

And

to

prove

this

implication

we

must

define

state

functions

ym

in

terms

of

the

variables

and

prove

the

implication

-LRB-

-RRB-

-RRB-

g

-LRB-

-RRB-

where

g

-LRB-

-RRB-

is

obtained

from

-LRB-

-RRB-

by

substituting

yi

for

all

the

free

occurrences

of

yi

in

-LRB-

-RRB-

for

i

m

The

collection

of

state

functions

ym

is

called

a

refinement

validity

of

the

implication

S

-LRB-

-RRB-

-RRB-

S

-LRB-

-RRB-

does

not

imply

the

existence

of

a

refinement

mapping

In

general

refinement

mappings

can

be

found

if

we

modify

the

specifications

by

adding

dummy

variables

-LSB-

-RSB-

The

Transformational

Framework

For

Fault-Tolerance

This

section

presents

a

brief

summary

of

the

transformational

approach

to

the

formal

treatment

of

untimed

fault-tolerant

programs

Faults

and

their

effects

Let

P

-LRB-

I

O

-RRB-

be

a

program

with

the

safety

specification

-LRB-

P

-RRB-

I

-LSB-

NP

-RSB-

v.

Informally

speaking

a

physical

fault

in

the

execution

of

P

causes

a

transition

from

a

valid

state

of

P

into

an

error

state

Continuation

of

the

execution

of

P

from

such

an

error

state

may

leads

to

a

failure

state

which

deviates

from

the

specification

of

P

Thus

in

general

a

physical

fault

can

be

modelled

as

an

effect

of

an

atomic

fault-operation

this

can

be

translated

to

an

action

formula

by

which

was

presented

in

Section

For

example

an

arbitrary

malicious

fault

may

set

the

variables

of

P

to

arbitrary

values

a

crash

in

a

processor

may

cause

the

variables

to

become

unavailable

and

a

fault

may

cause

the

loss

of

a

message

from

a

channel

Physical

faults

can

be

described

by

a

set

of

atomic

operations

F

called

a

fault-environment

which

interfere

with

the

execution

of

P

by

possibly

changing

the

values

of

variables

in

v.

F

can

be

specified

by

the

action

formula

NF

NF

-LRB-

o

-RRB-

Executing

P

-LRB-

I

O

-RRB-

on

a

system

with

a

fault-environment

F

is

simulated

by

interleaving

the

execution

of

the

operations

of

P

and

F

Therefore

interference

by

F

on

the

execution

of

P

can

be

defined

as

a

transformation

F

F

-LRB-

P

F

-RRB-

-LRB-

I

O

-LSB-

F

-RRB-

The

internal

and

external

behaviours

of

P

being

executed

on

a

system

with

faults

F

are

then

specified

respectively

by

-LRB-

F

-LRB-

P

F

-RRB-

-RRB-

I

-LSB-

NP

NF

-RSB-

v

and

-LRB-

F

-LRB-

P

F

-RRB-

-RRB-

I

-LSB-

NP

NF

-RSB-

v

The

fault-prone

properties

of

P

under

F

can

be

reasoned

about

in

terms

of

the

properties

of

F

-LRB-

P

F

-RRB-

We

call

F

-LRB-

P

F

-RRB-

the

F

affected

version

of

P

and

a

behaviour

of

F

-LRB-

P

F

-RRB-

an

F

affected

behaviour

of

P

Example

-LRB-

continued

-RRB-

For

the

processor-memory

interface

assume

that

the

memory

is

faulty

and

its

value

may

be

corrupted

Such

a

fault

can

be

represented

by

the

atomic

operation

fault

mem

-LRB-

mem

-RRB-

whose

action

formula

is

-LRB-

fault

-RRB-

-LRB-

mem

-RRB-

Let

F

ffaultg

The

F

affected

version

of

InterFace

is

F

-LRB-

InterF

ace

F

-RRB-

-LRB-

-LRB-

op

ready

-RRB-

fRp

W

p

Rm

Wm

faultg

-RRB-

Thus

-LRB-

F

-LRB-

InterF

ace

F

-RRB-

-RRB-

-LRB-

op

ready

-RRB-

-LSB-

-LRB-

Rp

-RRB-

-LRB-

W

p

-RRB-

-LRB-

Rm

-RRB-

-LRB-

Wm

-RRB-

-LRB-

fault

-RRB-

-RSB-

v

where

v

fop

val

memg

Fault-tolerance

Given

a

program

P

-LRB-

I

O

-RRB-

with

the

specification

S

-LRB-

P

-RRB-

-LRB-

I

-LSB-

NP

-RSB-

v

L

-RRB-

and

a

property

described

by

a

formula

P

is

said

to

implement

-LRB-

or

satisfy

-RRB-

if

S

-LRB-

P

-RRB-

-RRB-

Such

an

implementation

assumes

that

the

hardware

is

fault-free

i.e.

that

each

operation

of

P

is

executed

according

to

its

semantics

However

if

P

is

run

on

a

system

subject

to

hardware

faults

errors

may

occur

when

the

program

is

executed

and

lead

to

failures

which

violate

the

specification

S

That

is

when

a

set

F

of

faults

interferes

with

the

execution

of

P

the

F

affected

behaviours

may

not

satisfy

and

F

-LRB-

P

F

-RRB-

is

then

not

an

implementation

of

For

P

to

be

fault-tolerant

correcting

operations

must

be

carried

out

to

prevent

an

error

from

leading

to

a

failure

P

is

called

an

F

tolerant

implementation

of

if

F

-LRB-

P

F

-RRB-

is

an

implementation

of

S

-LRB-

F

-LRB-

P

F

-RRB-

-RRB-

-RRB-

This

means

that

the

behaviours

of

P

comply

with

the

specification

despite

the

presence

of

faults

F

i.e.

the

implementation

P

of

tolerates

the

faults

described

by

F

When

such

a

property

is

a

canonical

specification

of

a

program

Ph

S

-LRB-

Ph

-RRB-

Ih

-LSB-

NPh

-RSB-

u

Lh

Pl

is

called

an

F

tolerant

refinement

of

Ph

denoted

Pl

vF

Ph

if

Pl

is

an

F

tolerant

implementation

of

S

-LRB-

Ph

-RRB-

The

F

tolerant

refinement

relation

vF

is

stronger

than

the

ordinary

refinement

relation

i.e.

if

Pl

is

an

F

tolerant

refinement

of

Ph

then

Pl

is

a

refinement

of

Ph

but

the

converse

is

not

necessarily

true

F

tolerant

refinement

is

generally

not

reflexive

i.e.

P

vF

P

does

not

generally

hold

However

it

is

transitive

if

F

-LRB-

Pll

-RRB-

v

Pl

and

F

-LRB-

Pl

-RRB-

v

Ph

then

F

-LRB-

Pll

-RRB-

v

Ph.

This

allows

a

stepwise-style

development

of

a

fault-tolerant

program

Faults

-LRB-

or

their

representations

-RRB-

considered

at

a

lower

level

step

in

the

development

may

differ

from

those

in

the

previous

step

Note

that

this

definition

of

fault-tolerance

is

with

respect

to

the

specification

of

the

program

This

distinguishes

it

from

some

other

definitions

e.g.

-LSB-

-RSB-

where

P

is

F

tolerant

only

if

P

vF

P

It

is

important

to

point

out

that

it

seems

that

we

should

specify

both

fault

state

transitions

F

and

a

behavioural

fault

assumption

BF

about

F

-LRB-

or

local

and

global

fault

assumption

-LSB-

-RSB-

-RRB-

such

as

frequency

and

minimum

separation

of

the

faults

In

this

case

the

F

tolerant

refinement

of

Ph

by

Pl

should

be

proven

under

the

condition

BF

S

-LRB-

F

-LRB-

Pl

F

-RRB-

-RRB-

BF

-RRB-

S

-LRB-

Ph

-RRB-

Such

a

behavioural

fault

assumption

BF

is

in

general

a

safety

property

which

prevents

certain

fault

transitions

from

taking

place

This

implies

that

the

state

transition

relation

specification

-LSB-

NPl

NF

-RSB-

vl

BF

is

again

a

safety

property

and

hence

can

be

transformed

into

a

formula

of

the

form

-LSB-

NPl

-RSB-

vl

This

shows

that

S

-LRB-

F

-LRB-

Pl

F

-RRB-

-RRB-

BF

equals

S

-LRB-

F

-LRB-

Pl

-RRB-

-RRB-

for

some

and

therefore

BF

can

be

encoded

into

the

model

of

fault

operations

The

separation

of

fault

transitions

and

behavioural

fault

assumptions

usually

makes

it

easier

to

specify

the

F

affected

behaviours

of

program

Pl

To

prove

that

Pl

is

an

F

tolerant

refinement

of

Ph

it

is

however

helpful

to

obtain

a

canonical

form

Il

-LSB-

N

-RSB-

vl

Ll

so

that

the

proof

rule

for

refinement

given

in

-LSB-

-RSB-

can

be

directly

used

Example

-LRB-

continued

-RRB-

Assume

we

must

prove

that

the

fault-free

memory

InterFace

can

be

implemented

using

three

faulty

memories

whose

values

may

be

corrupted

under

the

condition

that

at

any

time

at

most

one

can

be

corrupted

Let

and

denote

the

current

values

of

the

three

memories

respectively

Each

memory

is

faulty

and

its

value

memi

may

be

corrupted

by

faulti

i

Let

fi

be

a

variable

with

value

space

and

assume

that

memi

has

been

corrupted

if

fi

Each

faulti

can

be

specified

as

follows

-LRB-

faulti

-RRB-

-LRB-

memi

-RRB-

-LRB-

f

i

-RRB-

F

NF

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

BF

-LRB-

-RRB-

Each

faulti

for

i

can

be

represented

as

the

operation

faulti

-LRB-

memi

fi

-RRB-

-LRB-

mem

f

-RRB-

-LRB-

mem

memi

f

-RRB-

The

specification

of

a

program

Pl

which

is

the

F

tolerant

refinement

of

InterFace

can

be

obtained

by

specifying

its

variables

initial

condition

operations

and

fairness

conditions

The

variables

and

initial

condition

are

as

follows

vl

fop

val

Il

-LRB-

op

ready

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

To

specify

the

operations

of

Pl

we

first

define

the

following

auxiliary

function

vote

-LRB-

x

y

z

-RRB-

-LRB-

x

if

x

y

or

x

z

y

if

x

y

and

x

z

The

specification

of

the

rest

of

the

program

is

-LRB-

Rpl

-RRB-

-LRB-

op

ready

-RRB-

-LRB-

read

-RRB-

-LRB-

W

pl

-RRB-

-LRB-

op

ready

-RRB-

-LRB-

write

-RRB-

-LRB-

Z

-RRB-

-LRB-

Rml

-RRB-

-LRB-

op

read

-RRB-

-LRB-

ready

-RRB-

-LRB-

vote

-LRB-

-RRB-

-LRB-

Wm

l

-RRB-

-LRB-

op

write

-RRB-

-LRB-

ready

-RRB-

-LRB-

-LRB-

val

-RRB-

-RRB-

-LRB-

-LRB-

f

i

-RRB-

-RRB-

NPl

-LRB-

Rp

l

-RRB-

-LRB-

W

p

l

-RRB-

-LRB-

Rml

-RRB-

-LRB-

Wm

l

-RRB-

Ll

WF

-LRB-

-LRB-

Rml

-RRB-

-RRB-

WF

-LRB-

-LRB-

Wm

l

-RRB-

-RRB-

-LRB-

Pl

-RRB-

Il

-LSB-

NPl

-RSB-

vl

Ll

S

-LRB-

Pl

-RRB-

-LRB-

Pl

-RRB-

We

define

the

following

operations

Rp

l

-LRB-

op

ready

-RRB-

op

read

W

pl

-LRB-

op

ready

-RRB-

-LRB-

op

val

-RRB-

-LRB-

w

v

-RRB-

-LRB-

w

write

v

Z

-RRB-

Rml

-LRB-

op

read

-RRB-

-LRB-

op

val

-RRB-

-LRB-

ready

vote

-LRB-

-RRB-

-RRB-

Wm

l

-LRB-

op

write

-RRB-

-LRB-

op

-LRB-

memi

-RRB-

-LRB-

fi

-RRB-

-RRB-

-LRB-

ready

-LRB-

val

-RRB-

-LRB-

-RRB-

-RRB-

where

-LRB-

memi

-RRB-

is

an

abbreviation

for

vector

-LRB-

-RRB-

-LRB-

fi

-RRB-

for

vector

-LRB-

-RRB-

-LRB-

val

-RRB-

for

vector

-LRB-

val

val

val

-RRB-

and

-LRB-

-RRB-

for

vector

-LRB-

-RRB-

Let

Pl

be

defined

as

Pl

-LRB-

Il

fRpl

W

pl

Rml

Wm

l

g

-RRB-

The

action

formulas

of

Rpl

W

pl

Rml

and

Wm

l

are

respectively

-LRB-

Rml

-RRB-

-LRB-

W

pl

-RRB-

-LRB-

Rml

-RRB-

and

-LRB-

Wm

l

-RRB-

Notice

that

S

-LRB-

F

-LRB-

Pl

F

-RRB-

-RRB-

Il

-LSB-

NPl

NF

-RSB-

vl

Ll

We

can

prove

that

Pl

is

an

F

tolerant

refinement

of

the

fault-free

InterFace

under

the

condition

BF

To

do

so

we

first

calculate

-LSB-

NPl

NF

-RSB-

vl

BF

into

-LSB-

NPl

-RSB-

vl

where

i

and

-LRB-

fi

fi

-RRB-

-LRB-

mem

fi

-RRB-

-LRB-

mem

f

-RRB-

-LRB-

mem

memi

fi

-RRB-

where

is

modulo

Define

the

following

mappings

from

vl

to

v

f

op

op

g

val

val

g

mem

vote

-LRB-

-RRB-

Using

the

proof

rules

of

TLA

we

then

can

then

easily

prove

the

implication

S

-LRB-

F

-LRB-

Pl

-RRB-

-RRB-

-RRB-

S

-LRB-

InterFace

-RRB-

which

shows

that

Pl

is

an

F

tolerant

refinement

of

InterFace

under

the

assumption

BF

Real-Time

and

Fault-Tolerance

This

section

shows

how

to

extend

the

framework

so

that

it

can

deal

with

both

real-time

and

fault-tolerant

properties

First

the

semantic

model

of

the

framework

has

to

be

extended

to

cope

with

the

specification

and

refinement

of

real-time

properties

Specification

and

refinement

of

real-time

programs

Deadlines

introduce

a

set

of

timing

constraints

over

a

program

requiring

its

operations

to

be

executed

neither

too

early

nor

too

late

for

example

to

prevent

the

loss

of

messages

in

a

one-element

buffer

messages

must

not

be

sent

to

it

too

fast

nor

removed

too

slowly

Let

time

be

represented

by

the

set

R

-LSB-

where

R

is

the

set

of

the

non-negative

real

numbers

To

describe

such

timing

constraints

each

atomic

operation

o

of

P

has

a

lower

time

bound

L

-LRB-

o

-RRB-

and

an

upper

time

bound

U

-LRB-

o

-RRB-

both

in

R

-LSB-

such

that

L

-LRB-

o

-RRB-

U

-LRB-

o

-RRB-

ffl

lower

bound

condition

for

each

operation

o

once

it

is

enabled

it

can

be

performed

only

if

it

has

been

continuously

enabled

for

at

least

L

-LRB-

o

-RRB-

time

units

ffl

upper

bound

condition

for

each

operation

o

it

must

not

be

continuously

enabled

for

more

than

U

-LRB-

o

-RRB-

time

units

without

being

performed

Thus

a

real-time

program

can

be

represented

as

a

triple

PT

hP

L

Ui

where

P

is

an

untimed

program

defined

in

the

previous

section

and

L

and

U

are

functions

from

the

atomic

operations

of

P

to

a

time

domain

such

that

L

-LRB-

o

-RRB-

U

-LRB-

o

-RRB-

for

any

operation

o

of

P

Example

-LRB-

continued

-RRB-

We

slightly

modify

the

processor-memory

interface

by

assuming

that

the

processor

issues

write

and

read

operations

alternately

and

periodically

first

ready

to

issue

a

write

operation

Assume

that

it

takes

the

memory

no

more

than

u

time

units

to

execute

an

issued

operation

to

completion

To

ensure

that

a

message

written

is

neither

repeatedly

read

nor

overwritten

before

being

read

it

is

required

that

the

period

d

of

issuing

an

operation

by

the

processor

be

greater

than

u

The

real-time

program

InterFaceT

hInterface

L

Ui

are

given

as

follows

I

-LRB-

next

write

-RRB-

-LRB-

op

ready

-RRB-

f

ready

to

issue

write

g

Rp

-LRB-

next

read

-RRB-

-LRB-

op

ready

-RRB-

-LRB-

op

ready

-RRB-

-LRB-

op

next

-RRB-

-LRB-

read

write

-RRB-

W

p

-LRB-

next

write

-RRB-

-LRB-

op

val

next

-RRB-

-LRB-

write

v

read

-RRB-

-LRB-

v

Z

-RRB-

Rm

-LRB-

op

read

-RRB-

-LRB-

op

val

-RRB-

-LRB-

ready

mem

-RRB-

Wm

-LRB-

op

write

-RRB-

-LRB-

op

mem

-RRB-

-LRB-

val

ready

-RRB-

O

fRp

W

p

Rm

Wmg

L

-LRB-

Rp

-RRB-

U

-LRB-

Rp

-RRB-

L

-LRB-

W

p

-RRB-

U

-LRB-

W

p

-RRB-

d

L

-LRB-

Rm

-RRB-

L

-LRB-

Wm

-RRB-

U

-LRB-

Rm

-RRB-

U

-LRB-

Wm

-RRB-

u

The

timed

interface

can

be

described

as

the

program

InterFaceT

h

-LRB-

I

O

-RRB-

L

Ui

As

for

untimed

programs

we

shall

require

an

exact

specification

-LRB-

PT

-RRB-

of

a

real-time

program

PT

We

first

introduce

into

each

program

a

distinguished

state

variable

now

to

represent

time

and

an

operation

to

advance

time

that

are

constrained

by

the

following

assumptions

-LSB-

-RSB-

ffl

time

starts

at

initially

now

ffl

time

never

decreases

-LSB-

-LRB-

now

-RRB-

-RSB-

now

ffl

time

R

-LRB-

now

t

-RRB-

These

three

assumptions

can

be

combined

to

specify

the

real-time

RT

-LRB-

now

-RRB-

-LSB-

-LRB-

now

-RRB-

-RSB-

now

R

-LRB-

now

t

-RRB-

We

also

assume

the

program

state

and

time

do

not

change

simultaneously

and

the

program

state

can

only

be

changed

by

the

program

operations

We

thus

rewrite

-LRB-

P

-RRB-

as

-LRB-

P

-RRB-

I

-LSB-

NP

-LRB-

now

-RRB-

-RSB-

v

Then

the

conjunction

-LRB-

P

-RRB-

RT

specifies

the

interleaving

program

operations

and

the

time

advancing

operations

divergence

is

also

called

No-Zeno

property

that

ensures

that

only

a

finite

number

of

operations

can

be

performed

in

any

finite

interval

of

time

The

program

operations

need

to

be

further

constrained

by

lower

bound

and

the

upper

bound

conditions

To

specify

the

lower

bound

L

-LRB-

o

-RRB-

for

each

operation

o

we

introduce

a

timer

to

which

is

an

auxiliary

state

variable

When

o

becomes

enabled

from

a

state

in

which

it

was

disabled

or

when

o

is

executed

to

is

assigned

a

clock

time

which

is

the

current

time

now

plus

L

-LRB-

o

-RRB-

units

of

time

Timer

-LRB-

to

L

-LRB-

o

-RRB-

v

-RRB-

-LRB-

-LRB-

go

to

L

-LRB-

o

-RRB-

-RRB-

-LRB-

go

to

-RRB-

-RRB-

-LSB-

-LRB-

-LRB-

-LRB-

o

-RRB-

go

-RRB-

L

-LRB-

o

-RRB-

go

-LRB-

o

-RRB-

to

-RRB-

-LRB-

v

-RRB-

-LRB-

now

-RRB-

-RSB-

-LRB-

to

v

-RRB-

Operation

o

can

not

be

executed

until

the

time

now

reaches

the

clock

time

to

MinTime

-LRB-

to

v

-RRB-

-LSB-

-LRB-

o

-RRB-

-RRB-

-LRB-

to

now

-RRB-

-RSB-

v

The

conjunction

Timer

-LRB-

to

L

-LRB-

o

-RRB-

v

-RRB-

MinTime

-LRB-

to

v

-RRB-

that

is

true

when

L

-LRB-

o

-RRB-

specifies

the

lower

bound

for

operation

o

and

the

lower

bound

condition

is

given

by

the

conjunction

of

the

lower

bound

conditions

for

all

the

operations

We

denote

this

conjunction

as

LB

-LRB-

PT

-RRB-

Similarly

to

specify

the

upper

bound

U

-LRB-

o

-RRB-

for

operation

o

a

timer

To

is

defined

by

Timer

-LRB-

To

U

-LRB-

o

-RRB-

v

-RRB-

obtained

by

substitution

in

the

definition

of

Timer

-LRB-

to

L

-LRB-

o

-RRB-

v

-RRB-

Operation

o

must

be

executed

before

the

time

now

passes

the

clock

time

To

MaxTime

-LRB-

To

-RRB-

-LSB-

To

-RSB-

now

The

conjunction

Timer

-LRB-

To

U

-LRB-

o

-RRB-

v

-RRB-

MaxTime

-LRB-

To

-RRB-

that

is

true

when

Up

-LRB-

o

-RRB-

gives

the

upper

bound

specification

for

each

operation

o

while

the

conjunction

of

the

upper

bound

specifications

for

all

the

operations

of

PT

is

the

specification

UB

-LRB-

PT

-RRB-

of

the

upper

bound

condition

The

time

bound

specification

B

-LRB-

PT

-RRB-

for

the

whole

program

PT

is

then

the

conjunction

LB

-LRB-

PT

-RRB-

UB

-LRB-

PT

-RRB-

The

real-time

executions

-LSB-

-LSB-

P

T

-RSB-

-RSB-

are

exactly

specified

by

-LRB-

PT

-RRB-

-LRB-

P

-RRB-

RT

B

-LRB-

PT

-RRB-

Hiding

the

internal

variables

x

and

the

auxiliary

timers

denoted

by

timer

-LRB-

PT

-RRB-

gives

the

canonical

specification

of

PT

-LRB-

PT

-RRB-

timer

-LRB-

PT

-RRB-

-LRB-

PT

-RRB-

To

prove

that

the

real-time

program

PT

satisfies

-LRB-

or

implements

-RRB-

a

timing

property

such

as

ffi

-LRB-

now

t

-RRB-

-LRB-

now

t

ffi

-RRB-

-RRB-

is

to

prove

the

implication

-LRB-

PT

-RRB-

-RRB-

ffi

Further

the

refinement

relation

v

between

real-time

programs

and

is

proved

as

the

implication

-LRB-

-RRB-

-RRB-

-LRB-

-RRB-

using

a

refinement

mapping

Example

-LRB-

continued

-RRB-

For

the

timed

processor-memory

interface

let

v

fnext

op

mem

valg

I

-LRB-

next

write

-RRB-

-LRB-

op

ready

-RRB-

-LRB-

Rp

-RRB-

-LRB-

next

read

-RRB-

-LRB-

op

ready

-RRB-

-LRB-

read

-RRB-

-LRB-

write

-RRB-

-LRB-

W

p

-RRB-

-LRB-

next

write

-RRB-

-LRB-

op

ready

-RRB-

-LRB-

write

-RRB-

-LRB-

read

-RRB-

-LRB-

Z

-RRB-

-LRB-

Rm

-RRB-

-LRB-

op

read

-RRB-

-LRB-

-LRB-

-RRB-

-LRB-

ready

mem

-RRB-

-RRB-

-LRB-

Wm

-RRB-

-LRB-

op

write

-RRB-

-LRB-

-LRB-

op

-RRB-

-LRB-

ready

val

-RRB-

-RRB-

LB

Timer

-LRB-

tRp

d

v

-RRB-

Timer

-LRB-

tW

p

d

v

-RRB-

MinTime

-LRB-

tRp

v

-RRB-

MinTime

-LRB-

tW

p

v

-RRB-

UB

Timer

-LRB-

TRp

d

v

-RRB-

Timer

-LRB-

TW

p

d

v

-RRB-

MaxTime

-LRB-

TRp

-RRB-

MaxTime

-LRB-

TW

p

-RRB-

Timer

-LRB-

TRm

u

v

-RRB-

Timer

-LRB-

TWm

u

v

-RRB-

MaxTime

-LRB-

TRm

-RRB-

MaxTime

-LRB-

TWm

-RRB-

Then

-LRB-

InterFaceT

-RRB-

-LRB-

next

write

-RRB-

-LRB-

op

ready

-RRB-

-LSB-

-LRB-

Rp

-RRB-

-LRB-

W

p

-RRB-

-LRB-

Rm

-RRB-

-LRB-

Wm

-RRB-

-RSB-

v

RT

LB

UB

-LRB-

InterfaceT

-RRB-

-LRB-

tRp

tW

p

TRp

TW

p

TRm

TWm

mem

-RRB-

-LRB-

InterfaceT

-RRB-

It

can

be

proved

that

that

InterfaceT

satisfies

the

property

-LRB-

op

read

mem

v

-RRB-

u

-LRB-

val

v

-RRB-

asserting

that

the

value

of

the

memory

will

be

output

within

u

units

of

time

after

the

processor

issues

an

read

operation

Another

property

of

the

program

is

:-LRB-

-LRB-

next

read

op

ready

mem

v

-RRB-

d

u

-LRB-

val

v

-RRB-

-RRB-

asserting

that

a

message

written

is

read

within

d

u

time

units

Faults

and

fault-tolerance

in

real-time

systems

The

functional

assumption

on

faults

is

still

modelled

by

a

set

F

of

atomic

operations

which

is

specified

by

the

action

formula

NF

We

do

not

impose

time

bounds

on

these

operations

Or

equivalently

we

assume

that

the

lower

bound

and

upper

bound

of

each

fault

operation

are

respectively

and

Given

a

real-time

program

PT

hP

L

Ui

the

F

affected

version

of

PT

is

then

defined

F

-LRB-

PT

F

-RRB-

hF

-LRB-

P

F

-RRB-

L

Ui

where

the

domain

of

L

and

U

are

extended

to

O

-LSB-

F

and

assign

each

operation

in

F

with

and

respectively

To

achieve

fault-tolerance

in

a

real-time

system

we

need

a

timing

assumption

on

the

occurrence

of

faults

especially

when

deadlines

are

required

to

be

met

Such

an

assumption

is

usually

a

constraint

on

the

frequency

of

the

occurrence

of

faults

or

the

minimal

period

of

time

for

which

faults

can

not

occur

This

period

of

time

should

be

long

enough

so

that

the

recovery

of

the

computation

can

take

place

and

progress

can

be

made

after

recovery

from

a

fault

The

timing

assumption

on

faults

F

is

a

conjunction

of

assumptions

each

of

which

has

a

form

such

as

whenever

occurs

can

not

occur

within

units

of

time

Remember

that

a

fault

say

is

specified

by

its

action

formula

-LRB-

-RRB-

The

above

timing

assumption

can

be

written

as

-LRB-

h

-LRB-

-RRB-

iv

-RRB-

-LSB-

-LRB-

-RRB-

-RSB-

v

-RRB-

A

formula

of

the

form

is

interpreted

as

must

always

hold

within

units

of

time

It

can

be

defined

in

in

terms

of

and

the

time

variable

now

as

-LRB-

now

t

-RRB-

-LRB-

now

t

-RRB-

-RRB-

-RRB-

Let

the

timing

assumption

on

faults

F

be

denoted

TF

the

internal

and

external

specifications

of

the

F

affected

version

F

-LRB-

PT

F

-RRB-

are

respectively

-LRB-

F

-LRB-

PT

F

-RRB-

-RRB-

I

-LSB-

-LRB-

NP

NF

-RRB-

-LRB-

now

-RRB-

-RSB-

v

RT

Bound

-LRB-

PT

-RRB-

BF

TF

-LRB-

F

-LRB-

P

F

-RRB-

-RRB-

RT

Bound

-LRB-

PT

-RRB-

TF

-LRB-

F

-LRB-

PT

F

-RRB-

-RRB-

timer

-LRB-

PT

-RRB-

-LRB-

F

-LRB-

PT

F

-RRB-

-RRB-

It

may

be

noticed

that

the

F

affected

version

of

a

real-time

program

PT

is

also

a

real-time

program

This

normal

form

allows

the

definition

of

fault-tolerance

for

real-time

systems

to

be

given

in

the

same

way

as

for

untimed

systems

A

real-time

program

PT

is

an

F

tolerant

implementation

of

a

real-time

property

if

the

implication

-LRB-

F

-LRB-

PT

F

-RRB-

-RRB-

-RRB-

holds

PT

is

an

F

tolerant

refinement

of

a

real-time

program

PTh

if

the

implication

holds

-LRB-

F

-LRB-

PT

F

-RRB-

-RRB-

-RRB-

-LRB-

PTh

-RRB-

holds

Example

-LRB-

continued

-RRB-

This

example

shows

how

the

timed

fault-free

processor-memory

interface

can

be

implemented

using

three

faulty

memories

whose

values

may

be

corrupted

Again

let

and

denote

the

current

values

of

the

three

memories

respectively

Each

memory

is

potentially

faulty

and

its

value

memi

may

be

corrupted

by

faulti

repetitively

where

i

Let

fi

be

an

auxiliary

variable

with

value

space

and

assume

that

memi

has

been

corrupted

if

fi

Each

faulti

can

be

specified

as

follows

for

i

-LRB-

faulti

-RRB-

-LRB-

memi

-RRB-

-LRB-

f

i

-RRB-

F

NF

-LRB-

-RRB-

-LRB-

-RRB-

-LRB-

-RRB-

Similar

to

the

untimed

case

the

specification

of

a

program

which

is

the

F

tolerant

refinement

of

InterFaceT

under

the

assumption

BF

that

no

more

than

one

copy

of

the

memory

are

corrupted.

can

be

obtained

as

the

variables

and

initial

condition

are

as

follows

fnext

op

val

-LRB-

next

op

-RRB-

-LRB-

write

ready

-RRB-

-LRB-

-RRB-

The

specification

of

the

operations

of

the

program

is

-LRB-

-RRB-

-LRB-

next

op

-RRB-

-LRB-

read

ready

-RRB-

-LRB-

-RRB-

-LRB-

write

read

-RRB-

-LRB-

W

-RRB-

-LRB-

next

op

-RRB-

-LRB-

write

ready

-RRB-

-LRB-

-RRB-

-LRB-

read

write

-RRB-

-LRB-

Z

-RRB-

-LRB-

-RRB-

-LRB-

op

read

-RRB-

-LRB-

ready

-RRB-

-LRB-

vote

-LRB-

-RRB-

-RRB-

-LRB-

Wm

-RRB-

-LRB-

op

write

-RRB-

-LRB-

ready

-RRB-

-LRB-

val

-RRB-

-LRB-

f

i

-RRB-

-LRB-

-RRB-

-LRB-

W

-RRB-

-LRB-

-RRB-

-LRB-

Wm

-RRB-

-LRB-

-RRB-

-LSB-

-RSB-

vl

-LRB-

Pl

-RRB-

-LRB-

Pl

-RRB-

Let

be

defined

as

-LRB-

W

Wm

g

-RRB-

where

action

formulas

of

W

and

Wm

are

respectively

-LRB-

-RRB-

-LRB-

W

-RRB-

-LRB-

-RRB-

and

-LRB-

Wm

-RRB-

To

meet

the

timing

properties

of

InterfaceT

it

requires

that

the

time

bounds

of

the

actions

of

the

implementation

guarantee

that

the

period

for

the

processor

to

issue

an

operation

is

still

d

and

the

upper

bound

for

the

memory

to

execute

an

issued

operation

to

completion

is

not

greater

than

u

u

Ll

-LRB-

-RRB-

Ul

-LRB-

Rpl

-RRB-

d

Ll

-LRB-

W

-RRB-

Ul

-LRB-

-RRB-

d

Ll

-LRB-

-RRB-

Ll

-LRB-

W

-RRB-

Ul

-LRB-

-RRB-

Ul

-LRB-

Wm

-RRB-

u

Then

we

can

prove

the

implication

-LRB-

F

-RRB-

BF

-RRB-

-LRB-

InterFaceT

-RRB-

that

asserts

that

F

tolerantly

implements

InterFaceT

under

the

assumption

BF

Note

that

the

assumption

can

be

relaxed

to

the

timing

assumption

where

-LSB-

TF

i

-LRB-

h

-LRB-

faulti

-RRB-

-RRB-

-LSB-

:-LRB-

-LRB-

faulti

-RRB-

-LRB-

faulti

-RRB-

-RRB-

-RSB-

-RRB-

asserting

that

only

one

of

the

most

recently

written

memory

may

be

corrupted

before

the

read

operation

is

completely

executed

Then

Ll

Uli

is

an

F

refinement

of

InterfaceT

under

the

faultassumption

TF

From

the

specifications

of

InterFacesT

and

we

can

see

a

practical

fact

to

achieve

fault-tolerance

with

timing

constraints

a

more

powerful

-LRB-

or

faster

-RRB-

machine

is

always

needed

The

execution

of

the

multiple

assignment

Wm

on

such

a

machine

should

not

be

slower

than

the

execution

of

the

single

assignment

Wm

on

a

machine

for

an

non-fault-tolerant

implementation

of

InterFaceT

and

the

execution

of

the

multiple

assignment

with

a

voting

function

should

not

be

slower

than

the

execution

of

the

single

assignment

Rm

Otherwise

with

a

machine

of

the

same

speed

the

original

time

bounds

must

be

loose

enough

to

give

enough

time

to

the

fault-tolerant

redundant

parts

This

point

becomes

even

clearer

when

we

refine

further

by

implementing

the

multiple

assignments

Wm

and

by

the

sequential

statements

respectively

Wm

s

-LRB-

op

write

-RRB-

val

val

val

op

ready

Rms

-LRB-

op

read

-RRB-

val

vote

-LRB-

-RRB-

op

ready

A

sequential

statement

can

be

specified

in

our

model

as

smaller

atomic

actions

by

introducing

control

variables

Let

cw

and

cr

be

two

variables

whose

value

domains

are

the

set

and

the

set

and

whose

initial

values

are

and

respectively

The

sequential

statement

Wm

s

can

be

broken

into

a

set

of

four

atomic

operations

fWm

Wm

Wm

Wm

which

are

specified

as

the

four

actions

-LRB-

Wm

-RRB-

-LRB-

op

cw

-RRB-

-LRB-

write

-RRB-

-LRB-

f

-RRB-

-LRB-

val

-RRB-

-LRB-

Wm

-RRB-

-LRB-

cw

-RRB-

-LRB-

f

-RRB-

-LRB-

val

-RRB-

-LRB-

Wm

-RRB-

-LRB-

cw

-RRB-

-LRB-

f

-RRB-

-LRB-

val

-RRB-

-LRB-

Wm

-RRB-

-LRB-

cw

-RRB-

-LRB-

-RRB-

-LRB-

ready

-RRB-

The

sequential

statement

Rms

is

sequentialisedinto

a

set

of

five

atomic

operations

in

the

same

way

as

Wm

s

has

been

The

newly

refined

program

denoted

as

Ps

has

an

enlarged

set

of

variables

vs

-LSB-

fcw

cr

and

a

strengthened

initial

condition

Is

cw

cr

Its

operations

are

Os

W

-LSB-

fWm

Wm

Wm

Wm

-LSB-

The

control

variables

fcw

cr

will

be

treated

as

internal

variables

Consider

the

time

bounds

of

this

program

to

meet

the

time

constraints

of

the

program

We

should

keep

the

time

bounds

of

and

W

unchanged

-LRB-

i.e.

both

the

lower

bound

and

upper

bound

are

still

d

-RRB-

And

we

require

that

the

sum

of

the

upper

bounds

of

the

four

operations

in

Wm

s

should

not

be

greater

than

the

upper

bound

of

Wm

and

the

sum

of

the

upper

bounds

of

five

operations

in

Rms

should

not

be

greater

than

the

upper

bound

of

Us

-LRB-

Rmsi

-RRB-

oei

oei

Us

-LRB-

Wm

sj

-RRB-

j

i

The

refinement

and

fault-tolerance

can

be

proven

by

showing

validity

of

the

implication

-LRB-

F

-LRB-

PTs

F

-RRB-

-RRB-

TF

-RRB-

-LRB-

F

-LRB-

F

-RRB-

-RRB-

TF

Recalling

that

is

an

F

tolerant

refinement

of

InterFaceT

PTs

is

also

an

F

refinement

of

InterFaceT

under

the

assumption

TF

Discussion

A

detailed

discussion

of

the

application

of

the

transformational

framework

for

fault-tolerance

can

be

found

in

earlier

publications

In

-LSB-

-RSB-

it

was

shown

how

an

F

tolerant

implementation

of

a

program

P

is

achieved

by

a

recovery

transformation

R

-LRB-

P

-RRB-

This

transformation

together

with

refinement

transformations

can

be

used

for

formal

treatment

of

most

existing

fault-tolerant

mechanisms

such

as

multipleimplementation

-LRB-

N-version

implementation

-RRB-

backward

and

forward

recovery

algorithms

-LSB-

-RSB-

recovery

blocksand

conversation

structures

for

fault-tolerance

-LSB-

-RSB-

-LSB-

-RSB-

formally

dealt

with

fault-tolerance

in

asynchronouscommunicating

systems

using

checkpointing

and

recovery

permitting

faults

also

during

checkpointing

and

recovery

-LSB-

-RSB-

presented

a

stepwise

and

modular

method

for

the

development

of

fault-tolerant

reactive

systems

it

showed

how

to

design

a

component

of

a

reactive

system

to

tolerate

both

software

design

faults

and

the

hardware

faults

of

a

given

component

It

also

dealt

with

fault-tolerant

broadcasts

-LSB-

-RSB-

and

Byzantine

agreement

-LSB-

-RSB-

The

earlier

work

did

not

deal

with

real-time

but

this

could

be

added

in

a

similar

way

to

that

shown

here

Suppose

that

fault-tolerance

was

to

be

achieved

using

checkpointing

and

backward

recovery

As

proposed

in

-LSB-

-RSB-

this

can

be

done

by

a

transformation

of

a

non-fault-tolerant

program

P

hI

Oi

to

a

program

R

-LRB-

P

-RRB-

hI

O

-LSB-

C

-LSB-

Ri

which

adds

to

P

the

checkpointing

operations

C

and

the

recovery

operation

R

When

a

real-time

program

PT

hP

L

Ui

is

considered

we

need

to

treat

the

time

constraints

on

the

actions

of

R

-LRB-

P

-RRB-

and

consider

real-time

program

R

-LRB-

P

-RRB-

T

hR

-LRB-

P

-RRB-

For

the

latter

to

be

an

F

tolerant

implementation

of

the

former

the

implication

-LRB-

F

-LRB-

R

-LRB-

P

-RRB-

T

F

-RRB-

-RRB-

TF

-RRB-

-LRB-

PT

-RRB-

www

Time

ccc

frrfrr

ccc

ccc

Execution

Of

PT

Execution

of

F

-LRB-

R

-LRB-

P

-RRB-

T

F

-RRB-

Figure

Relation

between

the

execution

of

PT

and

F

-LRB-

R

-LRB-

P

-RRB-

T

F

-RRB-

must

be

proved

for

some

time

assumption

TF

about

the

faults

F

Intuitively

and

TF

together

constrain

the

behaviour

in

the

following

way

The

time

bounds

on

C

ensure

that

checkpoints

are

taken

frequently

enough

to

limit

the

amount

of

computation

to

be

undone

after

error-recovery

But

a

high

frequency

of

checkpointing

also

adds

to

the

overhead

The

time

bounds

on

R

guarantee

that

whenever

a

fault

occurs

the

recovery

algorithm

R

executes

to

completion

within

a

small

time

r.

TF

assumes

that

the

system

becomes

stable

for

a

period

which

is

sufficient

for

the

execution

of

the

recovery

algorithm

R

and

for

some

progress

in

underlying

execution

of

P

so

that

the

original

time

constraints

of

P

can

be

met

The

relation

between

the

execution

of

the

original

real-time

program

PT

on

a

fault-free

system

and

the

execution

of

the

fault-tolerant

transformed

program

R

-LRB-

P

-RRB-

T

on

a

system

with

faults

F

is

depicted

in

Figure

-LSB-

-RSB-

where

c

is

for

checkpointing

operation

w

for

wasted

computation

r

for

recovery

operation

and

f

for

fault

operation

In

practice

periodic

checkpointing

can

be

used

and

it

can

be

assumed

that

faults

may

occur

with

some

minimum

separation

The

method

for

verification

of

fault-tolerance

and

real-time

properties

was

illustrated

using

a

simple

example

-LRB-

the

processor-memory

interface

-RRB-

However

for

programs

of

reasonable

size

some

mechanized

proof

checking

becomes

essential

Since

our

framework

does

not

need

a

new

semantic

model

there

is

also

no

need

for

a

new

proof

checking

tool

An

existing

tool

such

as

PVS

-LSB-

-RSB-

which

can

provide

support

for

temporal

reasoning

can

be

used

to

reason

about

fault-tolerance

-LSB-

-RSB-

gives

an

example

of

the

use

of

PVS

for

verifying

Byzantine

fault-tolerant

clock

synchronization

Conclusions

The

main

purpose

of

this

paper

was

to

show

how

using

transformations

stepwise

refinement

-LRB-

e.g.

-LSB-

-RSB-

-RRB-

can

be

used

for

the

development

of

fault-tolerant

systems

with

or

without

real-time

constraints

there

is

no

need

to

develop

a

new

semantics

for

dealing

with

fault-tolerant

systems

This

was

demonstrated

here

using

a

simple

action

system

and

TLA

but

the

idea

of

using

transformations

for

fault-tolerance

serves

equally

well

in

other

formal

frameworks

e.g.

-LSB-

-RSB-

uses

Hoare

's

CSP

-LSB-

-RSB-

extends

the

semantics

of

Hooman

's

work

in

-LSB-

-RSB-

for

fault-tolerance

with

real-time

and

-LSB-

-RSB-

uses

Milner

's

CCS

both

for

fault-tolerance

The

great

advantage

of

this

approach

is

that

developments

in

specification

and

verification

techniques

for

programs

can

be

used

also

for

fault-tolerant

systems

Acknowledgements

The

second

author

acknowledges

the

support

of

EPSRC

research

grants

and

References

-LSB-

-RSB-

M.

Abadi

and

L.

Lamport

The

existence

of

refinement

mapping

In

Proc

IEEE

Symposium

on

Logic

and

Computer

Science

-LSB-

-RSB-

M.

Abadi

and

L.

Lamport

Composing

specifications

Technical

Report

Digital

SRC

California

-LSB-

-RSB-

R.J.R.

Back

Refinement

calculus

Part

II

Parallel

and

reactive

programs

In

Lecture

Notes

in

Computer

Science

pages

Springer-Verlag

-LSB-

-RSB-

J.

Coenen

and

J.

Hooman

A

compositional

semantics

for

fault-tolerant

real-time

systems

In

J.

Vytopil

editor

Formal

Techniques

in

Real-Time

and

Fault-Tolerant

Systems

LNCS

pages

Springer-Verlag

-LSB-

-RSB-

J.

Coenen

and

J.

Hooman

Parameterized

semantics

for

fault-tolerant

real-time

systems

In

J.

Vytopil

editor

Formal

Techniques

in

Real-Time

and

Fault-Tolerant

Systems

pages

Kluwer

Academic

Publishers

-LSB-

-RSB-

E.

Diepstraten

and

R.

Kuiper

Abadi

lamport

towards

a

proof

theory

for

stuttering

dense

domains

and

refinement

mappings

In

Proceedings

of

the

REX

Workshop

on

Stepwise

Refinement

of

Distributed

Systems

Models

Formalisms

Correctness

LNCS

pages

Springer-Verlag

-LSB-

-RSB-

M.J.

Fischer

N.

D.

Griffeth

and

N.

A.

Lynch

Global

states

of

a

distributed

system

IEEE

Transactions

on

Software

Engineering

-LRB-

-RRB-

May

-LSB-

-RSB-

V.

Hadzilacos

An

algorithm

for

minimising

rollback

cost

In

Proceedings

of

ACM

Symposium

on

Principles

of

Database

Systems.

March

-LSB-

-RSB-

T.

Henzinger

Z.

Manna

and

A.

Pnueli

Temporal

proof

methodologies

for

real-time

systems

In

Proceedings

of

the

ACM

Annual

Symposium

on

Principles

of

Programming

Languages

pages

-LSB-

-RSB-

J.

Hooman

Specification

and

Compositional

Verification

of

Real-Time

Systems

LNCS

Springer-Verlag

-LSB-

-RSB-

T.

Janowski

Bisimulation

and

Fault-Tolerance

PhD

thesis

Department

of

Computer

Science

University

of

Warwick

-LSB-

-RSB-

K.H.

Kim

An

implementation

of

a

programmer-transparent

scheme

for

coordinating

concurrent

processes

in

recovery

In

pages

-LSB-

-RSB-

K.H.

Kim

Approach

to

mechanisation

of

the

conversation

scheme

based

on

monitors

IEEE

Transactions

on

Software

Engineering

-LRB-

-RRB-

May

-LSB-

-RSB-

R.

Koo

and

S.

Toueg

Checkpointing

and

rollback-recovery

for

distributed

systems

IEEE

Transactions

on

Software

Engineering

-LRB-

-RRB-

-LSB-

-RSB-

L.

Lamport

A

temporal

logic

of

actions

Technical

report

Digital

SRC

California

April

-LSB-

-RSB-

Z.

Liu

Fault-Tolerant

ProgrammingBy

Transformations

PhD

thesis

Department

of

Computer

Science

University

of

Warwick

-LSB-

-RSB-

Z.

Liu

and

M.

Joseph

Transformation

of

programs

for

fault

tolerance

Formal

Aspects

of

Computing

-LRB-

-RRB-

-LSB-

-RSB-

Z.

Liu

and

M.

Joseph

Specifying

and

verifying

of

recovery

in

asynchronous

communicating

systems

In

J.

Vytopil

editor

Formal

Techniques

in

Real-Time

and

Fault

Tolerant

Systems

pages

Kluwer

Academic

Publishers

-LSB-

-RSB-

Z.

Liu

and

M.

Joseph

Stepwise

development

of

fault-tolerant

reactive

systems

In

Formal

Techniques

in

Real-Time

and

Fault-Tolerant

Systems

LNCS

pages

Springer-Verlag

-LSB-

-RSB-

L.

Lamport

R.

Shostak

and

M.

Pease

The

Byzantine

General

problem

ACM

Transactions

on

Programming

Languages

and

Systems

-LRB-

-RRB-

July

-LSB-

-RSB-

J.M.

Morris

A

theoretical

basis

for

stepwise

refinement

and

the

programming

calculus

Science

of

Computer

Programming

-LSB-

-RSB-

C.

Morgan

Programming

From

Specifications

Prentice

Hall

International

Series

in

Computer

Science

-LSB-

-RSB-

J.

Nordahl

Specification

and

Design

of

Dependable

Communicating

Systems

PhD

thesis

Department

of

Computer

Science

Technical

University

of

Denmark

-LSB-

-RSB-

S.

Owre

J.

Rushby

and

N.

Shankar

PVS

A

prototype

verification

system

In

Proc

Conference

on

Automated

Deduction

pages

Springer-Verlag

-LSB-

-RSB-

D.

Peled

and

M.

Joseph

A

compositional

framework

for

fault-tolerance

by

specification

transformation

Theoretical

Computer

Science

-LSB-

-RSB-

B.

Randell

System

structure

for

software

fault

tolerance

IEEE

Transactions

on

Software

Engineering

-LRB-

-RRB-

June

-LSB-

-RSB-

B.

Randell

P.A.

Lee

and

P.C.

Treleaven

Reliability

issues

in

computing

systems

design

Computing

Survey

-LRB-

-RRB-

-LSB-

-RSB-

D.L.

Russell

State

restoration

in

systems

of

communication

processes

IEEE

Transactions

on

Software

Engineering

-LRB-

-RRB-

March

-LSB-

-RSB-

F.B.

Schneider

Fault-tolerant

broadcasts

ACM

Transactions

on

Programming

Languages

and

Systems

-LRB-

-RRB-

April

-LSB-

-RSB-

F.B.

Schneider

and

D.

Gries

Fault-tolerant

broadcasts

Science

of

Computer

Programming

-LSB-

-RSB-

N.

Shankar

Verification

of

real-time

systems

using

PVS

In

Proc

of

Computer

Aided

Verification

LNCS

pages

Springer-Verlag

-LSB-

-RSB-

Y.

Tamir

and

C.H.

Sequin

Error

recovery

in

multicomputers

using

global

checkpoints

In

Proceedings

of

International

Conference

on

Parallel

Processing

August

-LSB-

-RSB-

D.G.

Weber

Fault

tolerance

as

self-simlarity

In

J.

Vytopil

editor

Formal

Techniques

in

RealTime

and

Fault

Tolerant

Systems

pages

Kluwer

Academic

Publishers

-LSB-

-RSB-

W.G.

Wood

A

decentralised

recovery

control

protocol

In

Proceedings

of

the

Annual

International

Symposium

on

Fault-Tolerant

Computing

June

fault-tolerance
fault-tolerant refinement
real-time