A

Survey

on

Kernel

Specification

and

Verification

Ahmed

Helmy

email

ahelmy@usc.edu

May

Abstract

Formal

methods

have

been

traditionally

used

to

model

and

verify

operating

systems

Different

methods

verify

different

operating

systems

properties

such

as

process

management

mutual

exclusion

and

inter-process

communication

Moreover

various

methods

may

capture

different

design

errors

such

as

deadlocks

or

unspecified

receptions

The

system

kernel

supports

higher-level

system

services

Hence

kernel

verification

is

essential

for

the

proper

operation

of

the

system

In

addition

providing

clear

kernel

specification

improves

the

interoperability

between

its

various

implementations

In

this

paper

we

describe

commonly

used

methods

for

kernel

specification

and

verification

Some

methods

provide

a

mathematical

model

and

use

logic

to

prove

properties

of

interest

These

include

PVS

and

Boyer-Moore

logic

Others

use

a

programming

language

to

simulate

the

system

then

apply

verification

tools

to

capture

system

errors

These

include

the

SPIN

tool

Distributed

operating

systems

are

susceptible

to

unexpected

failure

events

complicating

the

issue

of

system

robustness

This

issue

is

not

addressed

explicitly

by

traditional

verification

methods

We

present

a

new

STRESS

method

that

can

be

used

to

analyze

system

robustness

STRESS

is

based

on

a

simulation

framework

and

facilitates

fault

simulation

for

distributed

operating

systems

and

kernels

Finally

our

comparison

shows

that

some

of

the

methods

discussed

are

complementary

Thus

several

methods

may

be

used

to

obtain

better

fault

coverage

of

the

target

system

Introduction

In

general

kernels

are

modeled

as

communicating

finite

state

machines

-LRB-

FSM

-RRB-

The

definition

of

a

FSM

requires

a

set

of

machine

states

and

a

definition

of

transitions

on

these

states

-LSB-

-RSB-

In

our

case

kernel

operations

control

the

changes

to

the

machine

states

Goals

for

obtaining

a

kernel

model

are

manifold

First

to

provide

a

precise

documentation

by

defining

the

required

behavior

of

the

kernel

interface

and

ignoring

implementation

issues

Second

to

define

a

contract

between

kernel

users

and

implementors

This

enhances

the

program

's

portability

by

providing

unambiguous

statement

of

the

required

features

of

a

kernel

implementation

Third

to

facilitate

proof

of

correctness

of

application

programs

which

run

on

the

kernel

using

mechanical

proof

checkers

-LSB-

-RSB-

In

this

research

paper

a

survey

of

the

methods

used

commonly

to

verify

and

validate

kernel

specifications

and

implementations

is

presented

This

study

is

not

meant

to

be

exhaustive

rather

illustrative

with

emphasis

on

the

limitations

and

shortfalls

of

the

conventional

methods

In

addition

we

describe

a

new

method

based

upon

a

simulation

model

to

analyze

the

robustness

of

distributed

operating

systems

and

kernels

under

loss

and

failure

scenarios

We

call

our

method

Systematic

Testing

of

Robustness

by

Examination

of

Selected

Scenarios

-LRB-

STRESS

-RRB-

All

the

methods

described

share

a

common

ultimate

goal

that

is

to

test

correctness

aspects

of

the

In

general

correctness

implies

the

absence

of

errors

System

errors

are

discussed

next

System

Errors

There

are

two

types

of

errors

addressed

by

kernel

design

-LRB-

-RRB-

design

errors

and

-LRB-

-RRB-

operational

errors

-LSB-

-RSB-

Design

errors

address

-LRB-

a

-RRB-

safety

-LRB-

b

-RRB-

liveness

and

-LRB-

c

-RRB-

responsiveness

properties

of

a

system

Safety

properties

ensure

that

the

system

never

enters

undesirable

states

and

include

freedom

from

deadlocks

assertion

violations

improper

termination

and

unspecified

receptions

Liveness

properties

ensure

that

the

system

performs

its

intended

functions

with

respect

to

service

specification

and

include

detection

of

acceptance

cycles

-LRB-

those

that

pass

through

acceptance

states

-RRB-

and

absence

of

non-progress

cycles

-LRB-

those

that

do

not

pass

through

any

progress-state

-RRB-

Responsiveness

properties

include

timeliness

and

fault

tolerance

which

recovers

the

system

to

a

legal

state

to

resume

normal

execution

from

an

illegal

state

Operational

errors

relate

to

the

implementation

environment

Recovery

mechanisms

deal

with

transient

operational

errors

which

may

change

the

state

of

a

system

but

may

not

change

its

behavior

This

wide

spectrum

of

potential

errors

motivated

the

development

of

various

approaches

for

correct

system

design

A

high-level

classification

of

these

approaches

is

outlined

next

Taxonomy

The

main

issues

addressed

by

the

methods

discussed

in

this

paper

are

specification

and

verification

this

document

we

conveniently

interchange

system

and

kernel

since

the

kernel

is

the

target

system

of

this

study

Specification

is

the

process

of

describing

a

system

and

its

desired

properties

Formal

specification

uses

a

language

with

a

mathematically-defined

syntax

and

semantics

System

properties

include

functional

behavior

timing

behavior

performance

characteristics

or

internal

structure

Verification

goes

one

step

beyond

specification

to

analyze

a

system

for

the

desired

properties

The

two

approaches

for

verification

are

theorem

proving

and

model

checking

-LSB-

-RSB-

ffl

Theorem

proving

In

theorem

proving

system

properties

are

expressed

in

logic

formulas

defining

a

set

of

axioms

and

rules

In

contrast

to

model

checking

theorem

proving

can

deal

with

infinite

state

spaces

However

interactive

theorem

provers

require

human

intervention

and

hence

are

slow

and

error-prone

Theorem

proving

includes

model-based

and

logic-based

formalisms

Model-based

formalisms

such

as

Z

and

VDM

are

suitable

for

kernel

specifications

in

a

succinct

manner

but

lack

the

tool

support

for

effective

proof

of

properties

The

use

of

first

order

logic

allows

the

use

of

theorem

provers

such

as

Nqthm

but

may

result

in

specifications

that

are

difficult

to

read

Higher

order

logic

such

as

PVS

provides

expressive

power

to

provide

clear

descriptions

and

proof

capabilities

for

kernel

properties

We

shall

describe

the

application

of

Nqthm

and

PVS

to

kernel

verification

in

section

ffl

Model

checking

Model

checking

relies

on

building

a

finite

model

of

a

system

and

checking

that

a

desired

property

holds

It

is

usually

used

for

reachability

analysis

-LSB-

-RSB-

The

check

is

performed

as

an

exhaustive

search

and

may

suffer

from

the

state

explosion

problem

In

this

model

a

program

is

used

to

simulate

the

behavior

of

the

system

SPIN

and

STRESS

fall

under

this

category

and

are

described

further

in

section

For

brevity

we

shall

only

address

a

subset

of

these

methods

that

have

actually

been

used

for

modeling

and

verification

of

kernels

The

rest

of

the

paper

is

organized

as

follows

Section

discusses

the

surveyed

methods

with

the

corresponding

case

studies

A

comparison

of

the

methods

is

presented

in

section

Finally

section

provides

the

conclusion

of

this

study

Verification

Methods

In

this

section

we

discuss

individual

verification

tools

and

present

some

case

studies

on

kernel

verification

As

previously

discussed

verification

methods

include

theorem

proving

and

model

checking

Theorem

proving

is

presented

first

This

discusses

the

Nqthm

prover

with

a

case

study

on

Mach

-LSB-

-RSB-

and

the

PVS

system

with

a

case

study

on

a

real-time

kernel

-LSB-

-RSB-

Then

model

checking

methods

are

discussed

presenting

the

SPIN

tool

-LSB-

-RSB-

with

a

case

study

on

the

Harmony

kernel

-LSB-

-RSB-

Our

new

STRESS

method

concludes

this

section

Theorem

Proving

A

kernel

state

consists

of

entities

such

as

memory

objects

processes

and

threads

Axioms

constrain

the

relations

between

entities

in

a

valid

kernel

state

This

is

illustrated

by

the

following

case

studies

Mach

Math

model

using

Nqthm

This

case

study

investigates

a

simple

model

of

some

properties

of

the

Mach

kernel

using

the

BoyerMoore

logic

and

Nqthm

Nqthm

Nqthm

is

an

automated

reasoning

system

also

known

as

the

Boyer-Moore

Theorem

Prover

Nqthm

is

a

Common

Lisp

program

for

proving

mathematical

theorems

The

logic

of

Nqthm

is

a

quantifier-free

first

order

logic

with

equality

The

rules

of

inference

of

the

logic

are

those

of

propositional

logic

and

equality

with

the

addition

of

mathematical

induction

Using

Nqthm

logic

concepts

and

axioms

are

introduced

and

are

used

to

derive

lemmas

New

functions

must

satisfy

the

concepts

and

axioms

To

better

understand

how

such

logic

is

used

in

kernel

modeling

we

discuss

its

application

to

Mach

Mach

math

model

Mach

is

a

micro-kernel

that

minimizes

the

abstractions

provided

by

the

kernel

and

provides

a

rich

set

of

semantics

associated

with

these

abstractions

The

Mach

kernel

state

consists

of

entities

such

as

tasks

threads

ports

messages

pages

and

memory

objects

Entities

may

participate

in

relations

For

example

a

port

right

is

a

relation

involving

a

task

and

a

port

that

characterizes

a

task

's

capability

on

a

port

Axioms

place

constraints

on

the

relations

that

may

hold

in

a

kernel

state

-LCB-

for

instance

at

most

one

task

may

hold

a

receive

right

on

a

given

port

The

description

of

entities

relations

and

constraints

characterizes

a

legal

Mach

state

Kernel

requirements

are

expressed

as

a

collection

of

functions

predicates

and

axioms

A

recognizer

predicate

is

identified

for

each

Mach

entity

class

For

example

an

entity

x

that

satisfies

the

predicate

taskp

-LRB-

x

oe

-RRB-

is

a

task

in

state

oe

A

thread

is

recognized

by

the

predicate

threadp

-LRB-

x

oe

-RRB-

Following

are

four

axioms

defining

the

relations

between

tasks

and

threads

Axiom

Disjointness

of

taskp

and

threadp

taskp

-LRB-

x

oe

-RRB-

threadp

-LRB-

x

oe

-RRB-

Axiom

The

relation

task

thread

formalizes

thread

ownership

The

expression

task

thread

-LRB-

t

th

oe

-RRB-

holds

in

state

oe

if

th

is

a

thread

in

task

t.

taskp

-LRB-

t

oe

-RRB-

threadp

-LRB-

th

oe

-RRB-

task

thread

-LRB-

t

th

oe

-RRB-

Axiom

A

thread

can

only

be

owned

by

one

task

task

thread

-LRB-

th

oe

-RRB-

task

thread

-LRB-

th

oe

-RRB-

Axiom

The

set

of

threads

associated

with

task

t

is

expressed

as

threads

-LRB-

t

oe

-RRB-

A

thread

th

is

an

element

of

threads

-LRB-

t

oe

-RRB-

if

and

only

if

it

is

owned

by

t.

th

threads

-LRB-

t

oe

-RRB-

task

thread

-LRB-

t

th

oe

-RRB-

To

define

the

Mach

state

we

construct

the

closures

for

the

axioms

For

example

the

closure

of

Axiom

is

-LRB-

taskp

-LRB-

x

oe

-RRB-

threadp

-LRB-

x

oe

-RRB-

-RRB-

Definition

A

legal

Mach

state

can

be

defined

as

the

conjunction

of

the

closures

of

the

axioms

legal

state

-LRB-

oe

-RRB-

-LRB-

taskp

-LRB-

x

oe

-RRB-

threadp

-LRB-

x

oe

-RRB-

-RRB-

Kernel

transitions

-LRB-

e.g.

Mach

atomic

actions

-RRB-

must

preserve

legality

i.e.

produce

legal

state

from

legal

state

One

can

methodically

examine

all

the

relations

given

and

identify

the

desired

actions

in

each

class

which

can

be

derived

from

that

relation

This

procedure

however

is

not

entirely

algorithmic

since

within

each

class

practical

issues

must

be

addressed

Limitations

The

Mach

model

-LRB-

for

atomic

actions

and

locks

-RRB-

includes

more

than

relations

axioms

and

functions

The

fact

that

axiomatization

is

not

algorithmic

may

limit

the

use

of

Nqthm

Real-time

kernel

model

using

PVS

We

present

a

specification

analysis

of

a

real

time

kernel

using

the

PVS

-LRB-

Prototype

Verification

System

-RRB-

system

PVS

PVS

is

a

verification

system

that

is

a

specification

language

integrated

with

support

tools

and

a

theorem

prover

It

is

based

on

classical

higher

order

logic

extended

with

a

typing

system

The

following

case

study

gives

the

approach

of

PVS

to

kernel

specification

and

verification

Real-time

kernel

specification

Real-time

operating

systems

are

vital

for

safety

critical

systems

and

their

correctness

is

essential

for

the

integrity

of

such

systems

Here

we

present

a

simplified

specification

of

a

real-time

kernel

The

specification

includes

abstract

model

of

the

functional

and

timing

requirements

for

the

kernel

and

a

minimal

model

of

the

system

supported

The

kernel

requirements

include

no

dynamic

task

creation

-LRB-

for

simplify

of

analysis

-RRB-

asynchronous

IPC

via

shared

data

allowed

by

Ada

's

protected

objects

and

mutual

exclusion

provided

by

a

priority

protocol

The

properties

of

the

kernel

that

need

to

be

verified

are

-LRB-

a

-RRB-

the

highest

priority

task

is

executed

and

-LRB-

b

-RRB-

mutual

exclusion

is

maintained

for

protected

objects

The

approach

produces

a

high

level

kernel

specification

including

minimal

model

of

application

program

and

interactions

between

application

and

kernel

This

provides

a

description

of

the

kernel

functionality

and

application

properties

required

for

proper

operation

of

the

system

The

model

of

interactions

allows

global

system

properties

to

be

analyzed

with

the

assistance

of

a

theorem

prover

The

high

level

kernel

specification

comprises

of

the

timing

properties

the

kernel

environment

and

the

kernel

state

Timing

properties

are

expressed

using

real-time

logic

-LRB-

RTL

-RRB-

The

logic

is

based

on

an

eventaction

model

where

actions

are

time

consuming

sections

of

work

and

events

are

temporal

markers

corresponding

to

external

stimuli

the

start

and

end

of

actions

and

system

state

changes

The

basic

types

for

the

kernel

environment

include

TASK

protected

objects

-LRB-

PO

-RRB-

and

interrupts

The

state

of

a

task

can

be

either

ready

to

run

or

suspended

For

protected

objects

the

value

of

the

guard

for

each

PO

entry

is

updated

by

the

kernel

under

mutual

exclusion

Axioms

are

used

to

express

these

requirements

For

example

each

task

is

assigned

a

unique

base

priority

The

kernel

state

consists

of

state

variables

state

of

each

task

in

the

system

and

state

of

each

protected

object

The

kernel

operations

consist

of

Delay

operations

-LRB-

to

suspend

tasks

-RRB-

operations

on

protected

objects

and

the

Dispatcher

Interfaces

between

kernel

and

the

system

in

terms

of

kernel

state

and

defined

operations

The

state

of

the

system

is

the

state

of

the

kernel

combined

with

the

state

of

the

program

To

execute

the

given

operation

in

the

system

the

good-operation

predicate

is

evaluated

It

consists

of

the

conjunction

of

the

following

four

properties

-LRB-

a

-RRB-

no

potentially

blocking

operations

are

executed

by

a

task

within

a

protected

object

-LRB-

b

-RRB-

a

task

can

only

exit

from

a

protected

object

if

it

is

in

one

-LRB-

c

-RRB-

the

object

must

be

available

for

a

task

that

is

attempting

to

enter

it

and

-LRB-

d

-RRB-

the

object

from

which

a

task

is

exiting

must

be

the

last

one

that

it

entered

The

operation

of

the

system

is

modeled

as

a

finite

sequence

of

operations

Each

operation

takes

a

finite

bounded

time

to

execute

ensuring

the

termination

of

execution

Sequences

of

kernel

operations

are

modeled

by

the

type

HISTORY

and

then

the

state

of

the

kernel

after

such

a

sequence

is

defined

by

an

interpreter

function

Int

Int

takes

an

initial

state

and

a

history

and

returns

the

state

of

the

kernel

after

this

sequence

of

operations

Modeling

all

possible

states

in

which

the

system

may

exist

is

achieved

by

applying

the

interpreter

function

to

the

initial

system

state

with

all

possible

sequences

of

operations

Hence

the

validity

of

an

invariant

property

P

-LRB-

s

-RRB-

is

expressed

as

-LRB-

h

History

-RRB-

P

-LRB-

Int

-LRB-

Init

h

-RRB-

-RRB-

where

Init

is

the

initial

state

of

the

system

The

general

class

of

properties

that

can

be

analyzed

in

this

manner

are

those

that

can

be

expressed

as

an

invariant

on

the

system

state

Limitations

-LRB-

a

-RRB-

In

general

PVS

suffers

a

similar

limitation

to

that

of

Nqthm

which

is

the

definition

of

a

large

number

of

relations

and

axioms

-LRB-

b

-RRB-

PVS

is

a

large

complex

system

Learning

it

may

consume

a

long

time

and

requires

lots

of

expertise

-LRB-

c

-RRB-

Another

limitation

is

related

to

the

use

of

Ada

Ada

rendezvous

requires

two

processes

to

both

reach

synchronization

points

which

can

cause

unbounded

delays

and

is

not

easily

amenable

to

standard

analysis

techniques

Model

Checking

One

may

classify

model

checking

methods

into

logic

model-checker

and

on-the-fly

Logic

modelchecker

works

with

a

two-pass

verification

process

In

the

first

pass

the

basic

behavior

of

the

system

is

explored

and

an

abstract

representation

is

obtained

In

the

second

pass

the

representation

is

used

to

prove

or

disprove

the

system

correctness

On-the-fly

verification

works

with

one-pass

verification

process

It

stores

in

memory

minimum

information

needed

for

correctness

verification

hence

on-the-fly

systems

can

handle

larger

problem

sizes

and

faster

than

a

model

checker

SPIN

lies

in

this

category

and

is

discussed

next

Harmony

kernel

verification

using

SPIN

SPIN

SPIN

is

a

general

verification

tool

for

proving

correctness

properties

of

distributed

or

concurrent

systems

These

systems

interact

through

shared

memory

rendezvous

operations

or

buffered

message

passing

Problems

that

these

interactions

may

create

can

be

debugged

by

SPIN

Once

the

system

design

is

obtained

a

proof

of

its

correctness

can

be

provided

SPIN

uses

the

validation

modeling

language

PROMELA

which

is

based

on

Hoare

's

CSP

language

-LSB-

-RSB-

PROMELA

models

communicating

sequential

processes

which

may

be

created

dynamically

and

communicate

asynchronously

or

synchronously

by

messages

through

channels

SPIN

supports

the

following

features

-LRB-

a

-RRB-

Automated

verification

based

on

the

principles

of

reachability

analysis

-LSB-

-RSB-

with

the

use

of

space

reduction

techniques

-LRB-

b

-RRB-

Complexity

profiling

statistics

gathered

during

verification

can

be

used

to

identify

the

hot

spots

in

the

validation

model

-LRB-

c

-RRB-

Search

for

safety

and

liveness

properties

Safety

errors

captured

include

assertion

violation

invalid

end

states

unspecified

receptions

and

unreachable

code

segments

Also

liveness

properties

such

as

acceptance

cycles

and

non-progress

cycles

are

detected

Two

types

of

simulations

are

supported

-LRB-

i

-RRB-

random

simulation

that

uses

a

randomizer

for

nondeterministic

selection

of

the

process

scheduling

algorithm

and

-LRB-

ii

-RRB-

guided

simulation

where

an

error

trail

in

terms

of

states

or

transitions

is

followed

A

guided

simulation

can

only

be

performed

if

a

verification

run

was

done

first

that

revealed

an

error

in

the

system

SPIN

is

intended

to

be

both

a

testbed

for

the

development

and

evaluation

of

new

verification

techniques

and

to

present

an

environment

for

verification

of

concurrent

systems

Harmony

Harmony

is

a

real-time

multitasking

multiprocessor

operating

system

It

is

based

on

a

microkernel

and

system

servers

It

features

interrupts

priority

preemptive

scheduling

intertask

communication

-LRB-

send-receive-reply

scheme

-RRB-

and

dynamic

tasks

Mutual

exclusion

across

processor

boundaries

is

achieved

by

an

ownership

protocol

rather

than

locks

The

first

step

is

to

formalize

the

models

of

the

system

the

scenarios

and

the

properties

that

are

to

be

checked

Focus

is

given

on

deadlocks

livelocks

and

other

safety

properties

for

the

primitives

of

Harmony

PROMELA

channels

are

used

for

abstracting

intertask

communications

Then

exhaustive

verification

of

the

intertask

communication

and

task

management

features

of

Harmony

was

carried

out

by

model-checking

Following

we

discuss

modeling

and

verification

of

intertask

communication

and

task

management

I

-RRB-

The

intertask

communication

model

is

composed

of

the

following

modules

-LRB-

a

-RRB-

kernel

module

including

parameters

such

as

number

of

processors

and

definitions

of

each

part

of

the

system

and

scenarios

-LRB-

b

-RRB-

task

descriptor

module

including

the

task

state

its

correspondent

its

processor

and

its

send

queue

-LRB-

c

-RRB-

processor

module

models

the

interrupt

masking

primitives

Local

mutual

exclusion

relies

on

interrupt

masking

-LRB-

d

-RRB-

interrupt

model

uses

non-deterministic

scheduling

and

-LRB-

e

-RRB-

queuing

primitives

Verification

of

intertask

communication

includes

absence

of

deadlocks

or

livelocks

appropriate

return

code

and

checking

that

all

the

used

code

To

achieve

this

an

attempt

is

made

to

find

a

set

of

scenarios

that

exhibit

all

possible

communication

schemes

Reduction

techniques

can

be

used

to

reduce

the

number

of

scenarios

investigated

for

example

by

considering

system

symmetries

-LRB-

such

as

all

processors

are

considered

identical

-RRB-

II

-RRB-

Task

management

and

creation

may

be

dynamic

but

may

introduce

infinite

state

space

at

validation

time

Hence

a

bounded

number

of

application

tasks

is

scheduled

at

initization

The

Interprocessor

interrupt

handling

takes

priority

over

the

running

tasks

if

they

are

not

masked

Verification

is

attained

by

simulating

and

verifying

basic

scenarios

involving

the

use

of

kernel

primitives

Limitations

-LRB-

a

-RRB-

The

fact

that

the

validator

indicates

that

all

the

code

is

reached

does

not

prove

that

every

situation

in

the

code

was

reached

though

For

example

if

-LRB-

-RRB-

then

else

Four

possible

values

for

and

but

may

encounter

only

of

them

to

cover

the

code

-LRB-

b

-RRB-

It

takes

a

long

time

to

get

acceptable

models

for

validation

ScenarioGeneration

End-pointScenariosTopologyLoss

Failures

End

PointTracingStateAnnotated

Simulation

Tracing

Output

Analysis

End

PointErrors

TransitionCodeProfiling

SimulationSet-up

SpecificTracing

SimulationEngine

CodeTracingTransitionTracing

Errors

Figure

STRESS

method

block

diagram

overview

-LRB-

c

-RRB-

Even

with

a

partial

model

and

a

restricted

scenario

we

may

be

faced

with

a

state

explosion

Partial

order

reduction

technique

integrated

into

SPIN

ameliorates

this

problem

Nevertheless

high

rate

of

communication

interaction

obviates

the

benefit

of

reduction

techniques

-LRB-

d

-RRB-

When

the

problem

size

is

large

it

is

difficult

to

follow

the

path

that

lead

to

the

error

STRESS

One

of

the

shortfalls

of

the

methods

discussed

so

far

is

that

they

do

not

explicitly

support

fault

and

error

modeling

In

reality

systems

consist

of

various

software

and

hardware

components

The

failure

of

any

component

may

affect

the

correctness

of

the

overall

system

In

order

to

design

robust

and

correct

systems

it

is

crucial

to

consider

failure

scenarios

as

an

integral

part

of

the

system

design

This

motivates

our

next

method

which

we

call

STRESS

for

Systematic

Testing

of

Robustness

by

Examination

of

Selected

Scenarios

By

robustness

we

refer

to

the

ability

of

a

system

to

react

correctly

in

the

face

of

failures

For

brevity

we

shall

only

present

an

overview

of

STRESS

STRESS

is

based

on

a

simulation

framework

and

supported

by

a

set

of

verification

tools

The

main

approach

aims

to

capture

and

analyze

a

set

of

error

prone

scenarios

under

various

failure

and

loss

conditions

This

is

achieved

by

investigating

representative

parts

of

the

state

space

and

the

definition

of

error

conditions

As

shown

in

figure

the

three

main

stages

of

our

approach

are

-LRB-

a

-RRB-

scenario

generation

-LRB-

b

-RRB-

tracing

and

-LRB-

c

-RRB-

output

analysis

Scenario

generation

Scenarios

are

the

collection

of

topologies

and

sequences

of

events

-LRB-

input

stimuli

and

state

transitions

-RRB-

that

describe

the

simulation

context

Analysis

of

these

sequences

can

unveil

weaknesses

in

the

system

design

Elements

of

a

scenario

include

-LRB-

i

-RRB-

topology

is

the

system

infrastructure

including

the

communication

bus

-LRB-

es

-RRB-

and

interconnect

-LRB-

s

-RRB-

memory

registers

and

queues

Topology

modules

are

designed

with

full

controllability

over

failures

and

loss

For

example

a

queue

delivers

messages

to

a

programmable

loss

module

that

either

delivers

the

message

or

drops

it

-LRB-

ii

-RRB-

end-point

scenarios

are

the

combination

of

possible

actions

by

the

end

point

Depending

on

the

system

modeled

the

end

point

may

correspond

to

micro-actions

such

as

read/write

to

registers

or

macro-actions

such

as

exchanging

messages

between

user

tasks

We

adopt

the

latter

approach

in

general

where

the

end

point

is

the

user

task

To

reduce

the

number

of

end-point

scenarios

we

introduce

the

notion

of

representative

scenarios

These

are

simple

scenarios

that

cover

a

large

portion

of

the

state

space

of

-LRB-

part

of

-RRB-

the

kernel

under

study

A

scenario

filter

is

used

in

choosing

such

scenarios

removing

impractical

combinations

or

redundant

events

due

to

symmetry

or

equivalence

in

topology

-LRB-

iii

-RRB-

failures

include

-LRB-

a

-RRB-

loss

or

corruption

of

data

on

the

communication

bus

and

-LRB-

b

-RRB-

loss

of

state

kept

in

memory

or

in

registers

Tracing

Tracing

collects

information

during

simulation

Traces

are

used

in

the

output

analysis

stage

and

include

-LRB-

i

-RRB-

end-point

or

-LRB-

macro

-RRB-

traces

giving

information

about

end-point

events

such

as

a

system

call

and

its

returned

code

-LRB-

ii

-RRB-

state

transition

or

-LRB-

micro

-RRB-

traces

providing

fine

grain

information

about

the

kernel

detailed

transitions

such

as

reads

and

writes

to

registers

-LRB-

iii

-RRB-

specific

traces

for

given

message

type

call

type

or

component

For

example

one

may

want

to

trace

the

event

of

opening

raw

sockets

as

a

call

This

information

is

fed

back

into

the

scenario

generator

to

program

the

infrastructure

loss

modules

to

lose

such

a

call

in

further

simulations

This

guides

the

simulation

and

gives

controllability

over

the

loss

scenarios

-LRB-

vi

-RRB-

annotated

code

tracing

for

use

in

the

profiler

in

a

later

stage

Partial

tracing

of

key

points

is

allowed

reducing

the

time

and

space

required

to

trace

other

pieces

of

code

Output

Analysis

At

this

stage

the

data

traced

during

the

simulation

are

analyzed

for

errors

and

profiling

It

includes

the

following

elements

-LRB-

i

-RRB-

end-point

errors

are

extracted

from

the

end-point

trace

according

to

a

specified

error

condition

The

error

condition

may

involve

correctness

or

performance

For

example

a

correctness

error

may

indicate

the

loss

of

a

return

code

While

a

performance

error

may

indicate

the

arrival

of

the

return

code

after

a

time

limit

End-point

errors

are

typically

used

to

guide

the

analysis

of

transition

errors

in

the

kernel

-LRB-

ii

-RRB-

transition

errors

are

obtained

from

both

the

end-point

errors

and

the

transition

traces

After

an

end-point

error

is

discovered

the

trace

is

rolled-back

in

time

to

relate

that

error

to

an

error

in

the

kernel

transitions

-LRB-

iii

-RRB-

profiling

conveys

information

about

the

segments

of

annotated

code

that

were

not

executed

and

the

hot

spots

in

the

executed

code

This

is

similar

to

reachability

analysis

Once

the

simulation

environment

is

set-up

the

verifier

runs

the

simulations

feeds

back

the

specific

traces

then

analyzes

the

output

To

obtain

the

scenarios

causing

the

error

the

basic

steps

are

revisited

in

reverse

order

i.e.

from

the

output

analysis

to

the

tracer

then

to

the

scenario

generator

The

scenarios

obtained

can

be

further

used

to

stress

kernel

implementations

Limitations

-LRB-

a

-RRB-

Code

coverage

may

not

correspond

directly

to

state

coverage

and

is

not

accurate

as

has

been

shown

for

SPIN

-LRB-

b

-RRB-

May

suffer

explosion

in

state

space

and

number

of

failure

scenarios

Scenario

and

space

reduction

techniques

are

used

to

ameliorate

this

problem

Efficiency

of

such

techniques

depends

on

the

feedback

from

the

specific

trace

and

the

size

of

the

system

and

topology

-LRB-

c

-RRB-

If

the

error

trail

is

long

it

may

be

hard

to

follow

Nontheless

the

roll-back

technique

and

the

use

of

end-point

traces

guide

the

error

trace

Summary

and

Comparison

In

this

section

we

present

a

summary

and

comparison

for

general

properties

of

the

discussed

methods

Although

the

comparison

is

not

detailed

it

gives

an

outline

of

the

major

differences

we

have

inferred

from

the

study

Nqthm

PVS

SPIN

STRESS

Verification

formal

proof

formal

proof

reachability

analysis

reachability

analysis

-LRB-

first

order

logic

-RRB-

-LRB-

higher

order

logic

-RRB-

-LRB-

state

code

coverage

-RRB-

-LRB-

code

scenario

coverage

-RRB-

Used

for

Specification

Specification

Analysis

partial

Analysis

implementation

analysis

analysis

conformance

testing

stress

testing

Robustness

no

explicit

support

no

explicit

support

no

explicit

support

supports

loss

failure

analysis

Limitations

No

of

axioms

No

of

axioms

state

explosion

scenario

explosion

Reusability

low

low

moderate

-LRB-

channels

-RRB-

high

-LRB-

infrastructure

-RRB-

As

shown

in

the

table

formal

methods

-LRB-

e.g.

Nqthm

or

PVS

-RRB-

provide

precise

documentation

and

hence

can

be

used

for

specification

as

well

as

analysis

through

proofs

On

the

other

hand

SPIN

and

STRESS

are

mainly

used

for

analysis

and

testing

All

methods

have

limitations

For

SPIN

and

STRESS

reachability

analysis

suffers

from

the

state

space

explosion

Both

methods

however

develop

space

reduction

techniques

to

circumvent

this

problem

With

the

availability

of

more

computation

and

storage

resources

we

do

not

consider

this

a

severe

limitation

For

PVS

and

Nqthm

the

number

of

relations

and

axioms

becomes

a

problem

with

the

growth

of

the

system

For

Reusability

formal

methods

re-define

entities

relations

and

axioms

for

the

new

system

SPIN

provides

communication

channels

while

STRESS

provides

a

system

infrastructure

that

can

be

re-used

Conclusion

We

have

presented

various

systematic

methods

for

kernel

verification

These

methods

address

a

wide

array

of

system

errors

such

as

liveness

safety

responsiveness

and

operational

errors

Verification

methods

use

either

theorem

proving

or

model

checking

Theorem

proving

methods

be

used

for

specification

as

well

as

analysis

but

require

the

construction

of

mathematical

proofs

Model

checking

methods

are

used

in

analysis

and

testing

but

may

experience

state

explosion

problems

Our

comparison

shows

that

for

large

systems

no

single

method

can

be

used

effectively

to

specify

analyze

and

test

the

system

Integration

of

these

methods

in

a

useful

manner

may

solve

this

problem

but

requires

clear

understanding

of

the

methods

and

the

target

system

References

-LSB-

-RSB-

A.

Agarwal

and

J.

W.

Atwood

A

Unified

Approach

to

Fault-Tolerance

in

Communication

Protocols

Based

on

Recovery

Procedures

IEEE/ACM

Transactions

on

Networking

Vol

No.

pages

-LCB-

Oct.

-LSB-

-RSB-

W.

R.

Bevier

Kit

A

Study

in

Operating

System

Verification

IEEE

Transactions

on

Software

Engineering

Nov.

-LSB-

-RSB-

W.

R.

Bevier

and

L.

M.

Smith

A

Mathematical

Model

of

the

Mach

Kernel

Atomic

Actions

and

Locks

Computational

Logic

Inc.

Technical

Report

URL

page

Apr.

-LSB-

-RSB-

W.

R.

Bevier

and

L.

M.

Smith

A

Mathematical

Model

of

the

Mach

Kernel

Entities

and

Relations

Computational

Logic

Inc.

Technical

Report

URL

page

Apr.

-LSB-

-RSB-

T.

Cattel

Modelization

and

Verification

of

a

Multiprocessor

Real

Time

OS

Kernel

Proceedings

of

FORTE

Bern

Switzerland

URL

Oct.

-LSB-

-RSB-

K.

M.

Chandy

and

L.

Lamport

Distributed

Snapshots

Determining

Global

States

of

Distributed

Systems

ACM

Transactions

on

Computer

Systems

Vol

No.

pages

-LCB-

Feb.

-LSB-

-RSB-

E.

M.

Clarke

and

J.

M.

Wing

Formal

Methods

State

of

the

Art

and

Future

Directions

CMU

Computer

Science

Technical

Report

page

Aug.

-LSB-

-RSB-

S.

Fowler

and

A.

Wellings

Formal

Analysis

of

a

Real-Time

Kernel

Specification

presented

at

Uppsala

Sweden

URL

-LSB-

-RSB-

D.

M.

Goldschlag

Mechanically

Verifying

Concurrent

Programs

with

the

Boyer-Moore

Prover

IEEE

Transactions

on

Software

Engineering

Also

available

from

Computational

Logic

as

technical

report

URL

page

Sept.

-LSB-

-RSB-

C.

Hoare

Communicating

Sequential

Processes

Communications

of

the

ACM

Vol

No.

pages

-LCB-

Aug.

-LSB-

-RSB-

G.

Holzmann

Design

and

Validation

of

Computer

Protocols

Prentice

Hall

AT&T

Bell

Labs.

-LSB-

-RSB-

G.

J.

Holzmann

What

's

New

in

SPIN

version

Bell

Laboratories

New

Jersey

page

Nov.

-LSB-

-RSB-

H.

Liu

and

R.

E.

Miller

Generalized

Fair

Reachability

Analysis

for

Cyclic

Protocols

IEEE/ACM

Transactions

on

Networking

Vol

No

pages

-LCB-

Apr.

O O O B B O B O O O O O O O O O O B I O O O O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B B O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O B B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B I O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O B O O O B O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O B B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O B O O O O O B O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O B O O B B O B I O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O B O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

formal methods
kernel
operating systems
robustness
specification
verification